-- Logs begin at Sun 2022-09-18 05:44:00 UTC, end at Sun 2022-09-18 07:12:33 UTC. -- Sep 18 06:15:50.122392 dv-cinder-855359-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 18 06:15:50.131817 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 18 06:15:50 2022] *** Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: nodename: dv-cinder-855359-3-iscsi Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: machine: x86_64 Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: clock source: unix Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: pcre jit disabled Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: detected number of CPU cores: 4 Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: current working directory: / Sep 18 06:15:50.134528 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: detected binary path: /usr/bin/uwsgi-core Sep 18 06:15:50.135229 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: your processes number limit is 47680 Sep 18 06:15:50.135229 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: your memory page size is 4096 bytes Sep 18 06:15:50.135229 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: detected max file descriptor number: 2048 Sep 18 06:15:50.135229 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: lock engine: pthread robust mutexes Sep 18 06:15:50.135229 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: thunder lock: enabled Sep 18 06:15:50.135229 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 18 06:15:50.135229 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 18 06:15:50.173512 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: Python main interpreter initialized at 0x55e2aea129c0 Sep 18 06:15:50.173512 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: python threads support enabled Sep 18 06:15:50.173512 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: your server socket listen backlog is limited to 100 connections Sep 18 06:15:50.173512 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: your mercy for graceful operations on workers is 90 seconds Sep 18 06:15:50.173872 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: mapped 403077 bytes (393 KB) for 2 cores Sep 18 06:15:50.173947 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: *** Operational MODE: preforking *** Sep 18 06:15:50.174168 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: *** uWSGI is running in multiple interpreter mode *** Sep 18 06:15:50.174168 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: spawned uWSGI master process (pid: 105404) Sep 18 06:15:50.174477 dv-cinder-855359-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 18 06:15:50.175315 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: spawned uWSGI worker 1 (pid: 105405, cores: 1) Sep 18 06:15:50.176087 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: spawned uWSGI worker 2 (pid: 105406, cores: 1) Sep 18 06:15:50.176087 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 18 06:15:50.840122 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 18 06:15:50.843276 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 18 06:15:51.924707 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:15:51.924707 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: from cryptography.utils import int_from_bytes Sep 18 06:15:51.925314 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:15:51.925314 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: from cryptography.utils import int_from_bytes Sep 18 06:15:51.926288 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:15:51.926288 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: from cryptography.utils import int_from_bytes Sep 18 06:15:51.926737 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:15:51.926737 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: from cryptography.utils import int_from_bytes Sep 18 06:15:54.388398 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 06:15:54.388992 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 06:15:54.389158 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 06:15:54.389513 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 06:15:54.389763 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 06:15:54.389856 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 06:15:54.390213 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 06:15:54.390300 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.390450 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.390618 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 06:15:54.390794 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.390907 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 06:15:54.391147 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 06:15:54.391669 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.391842 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.391930 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.392166 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.392346 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.392501 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.392601 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.392803 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.392894 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.393112 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.393213 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.394966 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395090 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395090 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395230 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395308 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395386 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395458 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-855359-3-iscsi {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395530 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395602 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395687 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395687 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-855359-3-iscsi {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395819 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395892 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.395968 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396038 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396116 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396189 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396255 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396324 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396394 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396464 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396607 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.396727 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.397002 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.397342 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.397504 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.397779 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.397875 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.398087 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.398190 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.398393 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.399100 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.399278 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.399598 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.399946 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.400265 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.400573 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.400893 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.401030 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.401197 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.401365 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.401603 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-855359-3-iscsi {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.401743 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.401922 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.402019 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.402234 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.402326 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.402546 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.402706 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.402925 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.403018 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.403276 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.403361 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-855359-3-iscsi {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.403582 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.403673 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.403900 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.404002 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.404213 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.404297 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.404517 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.404614 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.404841 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.404939 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.405158 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.405250 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.405473 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.405571 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.405787 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.405880 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.406087 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.406174 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.406390 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.406492 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.406730 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.406842 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.407045 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.407142 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.407345 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.407439 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.407640 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.407731 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.407884 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.407975 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.408191 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.408295 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.408507 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.408600 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.408815 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.408901 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.409116 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.409209 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.409361 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.409462 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.409613 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.409705 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.409907 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.409994 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.410222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.410310 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.410531 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.410763 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.410849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.410929 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.411052 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.411208 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.411366 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.411550 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.411687 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.411827 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.411975 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.412121 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.412263 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.412406 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.412550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.412685 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.412818 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.412957 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.413100 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.413240 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.413373 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.413513 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.413675 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.413829 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.413975 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.414109 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.414888 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.415031 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.20 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.415140 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.415284 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.20 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.415457 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.415590 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.415759 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.415928 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.416073 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.416243 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.416463 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.20 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.416561 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.416709 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.20 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.416854 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.417009 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.417175 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.417340 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.417477 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.417652 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.417761 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.417954 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.418059 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.418268 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.418381 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.418595 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.418710 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.418901 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.418993 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.419200 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.419302 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.419500 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.419612 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.419809 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.419958 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.420124 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.420212 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.420427 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.420513 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.420658 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.420753 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.420957 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.421051 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.421256 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.421342 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.421574 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.421661 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.421877 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.421970 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.422175 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.422268 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.422475 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.422586 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.422798 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.422882 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.423097 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.423198 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.423396 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.423483 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.423695 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.423781 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.423996 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.424090 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.424296 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.424380 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.424594 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.424686 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.424893 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.424989 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.425192 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.425278 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.425491 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.425583 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.425787 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.425884 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.426087 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.426172 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.426398 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.426491 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.426720 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.426820 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.427018 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.427109 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.427315 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.427402 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.427614 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.427708 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.427915 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.427999 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.428211 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.428302 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.428508 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.428607 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.428807 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.428899 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.429106 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.429200 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.429404 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.429490 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.429705 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.429830 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.430027 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.430120 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.430325 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.430419 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.430649 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.430752 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.430947 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.431043 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.431246 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.431340 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.431546 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.431636 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.431845 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.431937 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.432143 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.432238 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.432439 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.432532 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.432733 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.432825 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.433031 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.433117 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.433329 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.433421 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.433625 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.433722 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.433923 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.434010 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.434217 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.434310 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.434526 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.434646 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.434863 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.434950 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.435166 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.435252 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.435465 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.435561 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.435764 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.435856 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.436076 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:15:54.436162 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.436406 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.436492 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.436730 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.436823 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.437036 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.437122 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.437353 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.437439 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.437677 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.437770 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.437983 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.438068 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.438292 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.438379 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.438622 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.438722 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.438922 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.439027 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.439236 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.439321 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.439539 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.439631 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.439842 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.439943 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.440148 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.440241 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.440458 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.440551 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.440764 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.440849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.441079 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.441175 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.441392 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.441476 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.441701 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.441797 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.442007 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.442098 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.442310 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.442401 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.442650 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.442751 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.442971 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.443059 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.443282 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.443368 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.443587 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.443683 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.443907 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.444001 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.444215 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.444303 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.444522 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.444619 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.444842 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.444927 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.445156 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.445262 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.445484 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.445575 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.445789 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.445887 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.446091 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.446185 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.446421 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.446509 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.446752 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.446842 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.447062 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.447148 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.447377 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.447513 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.447687 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.447816 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.447995 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.448083 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.448327 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.448415 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.448630 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.448751 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.448941 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.449063 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.449246 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.449360 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.449550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.449677 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.449864 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.449986 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.450175 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.450296 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.450484 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.450619 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.450815 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.450945 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.451118 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.451233 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.451422 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.451536 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.451725 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.451848 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.452028 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.452141 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.452389 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.452512 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.452617 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.452730 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.452907 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.453028 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.453206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.453318 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.453509 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.453624 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.453807 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.453928 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.454123 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.454210 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.454435 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.454530 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.454760 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.454847 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.455063 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.455148 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.455373 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.455469 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.455675 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.455761 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.455975 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.456060 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.456278 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.456365 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.456580 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.456666 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.456881 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.456986 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.457184 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.457276 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.457485 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.457614 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.457786 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.457885 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.458091 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.458177 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.458390 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.458475 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.458733 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.458860 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.459037 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.459135 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.459339 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.459440 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.459647 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.459732 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.459951 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.460037 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.460255 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.460331 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.460560 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.460638 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.460865 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.460940 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.461713 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.461796 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.462021 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.462099 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.463889 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.464504 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.464592 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.464690 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.464690 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.464837 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.464837 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.464978 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.464978 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465133 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465215 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465293 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465370 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465455 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465544 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465626 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465705 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465783 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465869 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.465956 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466045 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466132 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466213 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466291 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466368 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466454 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466547 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466675 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466768 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.466928 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.467005 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.467091 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.467170 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.467333 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.467422 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.467649 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.467749 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.467950 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.468037 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.468246 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.468341 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.468544 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.468709 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.468841 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.468928 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.469173 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.469269 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.469472 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.469572 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.469799 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.469892 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.470111 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.470213 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.470418 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.470514 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.470749 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.470837 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.471060 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.471147 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.471379 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.471477 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.471687 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.471773 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.472003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.472098 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.472306 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.472393 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.472613 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.472709 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.472917 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.473010 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.473220 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.473307 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.473521 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.473606 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.473824 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.473967 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.474141 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.474229 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.474444 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.474530 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.474782 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.474869 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.475082 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.475170 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.475392 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.475492 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.475703 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.475798 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.476013 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.476099 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.476321 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.476407 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.476711 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.477335 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.477511 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.477597 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.477824 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.477911 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.478130 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.478232 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.478435 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.478521 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.478767 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.478867 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.479073 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.479173 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.479386 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.479472 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.479690 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.479782 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.479991 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.480086 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.480293 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.480379 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.480610 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.480696 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.480913 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.481001 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.481224 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.481318 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.481522 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.481608 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.481832 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.481928 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.482139 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.482228 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.482448 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.482534 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.482780 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.482869 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.483084 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.483176 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.483388 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.483483 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.483692 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.483788 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.483997 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.484134 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.484304 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.484398 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.484600 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.484693 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.484900 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.484984 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.485203 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.485296 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.485509 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.485594 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.485809 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.485896 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.486109 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.486222 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.486405 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.486501 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.486739 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.486829 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.487045 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.487144 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.487344 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.487430 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.487642 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.487727 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.487948 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.488035 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.488251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.488335 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.488550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.488635 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.488863 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.488955 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.489170 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.489279 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.489466 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.489572 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.489765 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.489852 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.490079 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.490164 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.490382 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.490475 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.490702 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.490794 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.491003 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.491092 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.491301 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.491396 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.491613 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.491698 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.491926 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.492019 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.492223 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.492357 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.492556 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.492644 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.492870 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.492955 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.493177 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.493277 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.493480 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.493574 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.493784 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.493870 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.494089 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.494219 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.494393 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.494485 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.494725 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.494814 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.495032 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.495120 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.495337 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.495424 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.495641 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.495734 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.495946 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.496045 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.496246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.496332 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.496553 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.496641 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.496854 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.496940 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.497160 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.497432 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.497518 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.497752 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.497909 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.498091 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.498186 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.498360 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.498447 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.498730 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.498943 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.499048 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.499270 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.499360 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.499578 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.499671 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.499892 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.499993 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.500228 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.500332 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.500543 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.500638 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.500844 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.500931 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.501146 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.501234 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.501455 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.501541 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.501756 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.501851 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.502059 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.502142 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.502360 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.502455 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.502685 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.502787 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.502992 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.503085 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.503289 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.503373 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.503587 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.503682 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.503886 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.503972 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.504192 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.504325 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.504489 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.504583 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.504785 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.504870 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.505080 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.505168 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.505379 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.505467 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.505686 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.505771 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.505980 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.506067 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.506278 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.506365 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.506617 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.506706 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.506912 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.507013 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.507215 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.507303 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.507511 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.507597 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.507809 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.507927 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.508133 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.508230 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.508469 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.508557 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.509030 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.509132 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.509358 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.509444 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.509659 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.509745 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.509961 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.510061 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.510263 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.510351 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.510569 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.510667 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.510907 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.511006 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.511204 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.511296 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.511505 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.511591 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.511802 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.511898 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.512105 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.512199 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.512406 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.512492 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.512703 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.512796 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.513006 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.513142 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.513303 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.513388 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.513603 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.513696 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.513903 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.513989 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.514206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.514300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.514484 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.514621 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.514806 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.514893 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.515108 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.515197 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.515407 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.515493 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.515703 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.515798 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.516000 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.516095 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.516302 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.516393 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.516600 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.516698 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.516893 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.516987 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.517190 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.517275 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.517482 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.517574 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.517791 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.517885 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.518086 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.518171 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.518385 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.518472 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.518713 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.518815 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.519016 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.519101 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.519315 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.519401 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.519611 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.519698 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.519909 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.519994 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.520207 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.520293 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.520503 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.520589 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.520799 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.520885 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.521101 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.521193 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.521399 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.521496 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.521701 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.521787 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.521998 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.522089 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.522298 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.522393 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.522626 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.522764 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.522928 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.523051 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.523228 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.523340 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.524044 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.524542 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.525042 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.525352 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.525492 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.525676 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.525825 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.525990 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.526147 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.526298 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.526468 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.526628 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.526816 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.526938 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.527129 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.527281 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.527438 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.527585 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.527746 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.527950 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.528046 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.528256 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.528391 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.528569 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.528664 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.528886 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.528983 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.529197 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.529283 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.529505 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.529592 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.529818 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.529913 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.530121 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.530208 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.530433 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.530519 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.530766 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.530872 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.531080 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.531167 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.531382 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.531470 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.531684 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.531772 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.531988 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.532073 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.532296 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.532428 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.532599 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.532687 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.532912 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.532998 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.533218 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.533305 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.533526 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.533650 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.533825 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.533919 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.534125 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.534212 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.534430 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.534525 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.534764 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.534853 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.535107 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.535204 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.535370 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.535456 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.535681 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.535792 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.535992 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.536097 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.536290 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.536594 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.536685 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.536897 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.536996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.537196 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.537285 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.537498 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.537587 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.537802 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.537901 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.538103 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.538192 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.538407 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.538497 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.538736 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.538841 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.539039 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.539128 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.539393 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.539491 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.539693 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.539784 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.539998 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.540086 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.540300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.540391 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.540599 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.540729 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.540915 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.541039 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.541220 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.541349 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.541520 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.541652 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.541824 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.541959 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.542124 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.542267 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.542429 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.542597 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.542756 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.542908 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.543177 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.543278 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.543485 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.543611 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.543791 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.543921 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.544092 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.544229 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.544398 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.544537 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.544702 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.544839 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.545010 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.545157 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.545317 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.545448 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.545617 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.545769 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.546025 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.546121 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.546330 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.546416 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.546657 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.546748 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.546963 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.547052 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.547262 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.547348 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.547561 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.547646 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.547957 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.548045 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.548278 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.548363 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.548579 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.548675 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.548876 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.548986 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.549174 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.549295 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.549490 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.549605 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.549790 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.549927 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.550106 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.550247 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.550515 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.550630 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.550848 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.550961 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.551149 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.551237 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.551476 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.551569 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.551798 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.551894 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.552119 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.552219 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.552426 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.552514 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.552740 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.552838 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.553043 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.553138 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.553347 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.553432 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.553656 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.553752 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.553960 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.554057 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.554278 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.554364 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.554623 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.554726 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.555433 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.555523 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.555744 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.555838 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.556045 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.556135 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.556363 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.556451 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.556680 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.556763 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.556989 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.557077 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.557305 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.557398 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.557606 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.557712 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.557913 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.558029 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.558234 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.558322 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.558545 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.558695 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.558877 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.558966 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.559189 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.559285 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.559496 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.559593 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.559800 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.559899 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.560103 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.560198 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.560419 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.560513 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.560721 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.560806 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.561023 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.561108 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.561329 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.561425 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.561634 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.561729 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.561939 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.562026 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.562240 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.562374 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.562538 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.562708 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.562880 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.562967 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.563183 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.563280 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.563489 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.563584 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.563791 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.563878 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.564106 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.564193 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.564418 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.564504 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.564717 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.564805 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.565034 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.565122 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.565341 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.565465 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.565640 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.565727 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.565940 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.566039 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.566240 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.566334 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.566597 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.566698 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.566903 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.566988 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.567216 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.567304 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.567537 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.567637 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.567839 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.567927 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.568140 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.568235 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.568453 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.568548 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.568754 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.568852 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.569057 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.569144 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.569359 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.569455 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.569663 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.569749 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.569962 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.570050 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.570262 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.570349 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.570664 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.570755 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.570979 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.571064 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.571280 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.571369 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.571580 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.571667 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.571881 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.571968 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.572183 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.572270 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.572479 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.572572 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.572777 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.572871 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.573079 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.573164 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.573394 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.573480 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.573691 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.573785 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.574007 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.574093 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.574310 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.574402 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.574626 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.574728 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.574931 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.575018 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.575228 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.575314 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.575555 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.575649 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.575850 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.575935 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.576166 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.576250 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.576468 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.576563 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.576774 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.576867 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.577085 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.577172 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.577393 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.577479 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.577699 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.577786 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.578007 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.578221 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.578316 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.578558 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.578677 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.578903 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.578991 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.579219 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.579317 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.579528 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.579612 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.579836 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.579924 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.580146 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.580245 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.580460 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.580663 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.580763 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.580989 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.581085 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.581302 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.581398 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.581612 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.581700 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.581921 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.582018 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.582230 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.582327 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.582539 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.582673 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.582872 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.582961 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.583183 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.583273 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.583489 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.583576 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.583795 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.583891 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.584116 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.584203 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.584433 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.584521 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.584753 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.584881 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.585063 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.585148 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.585378 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.585481 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.585685 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.585770 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.586500 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.586659 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.586832 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.586934 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.587148 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.587234 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.587463 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.587549 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.587765 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.587851 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.588071 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.588155 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.588378 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.588476 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.588686 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.588773 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.588991 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.589088 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.589309 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.589451 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.589602 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.589692 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.589907 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.589996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.590211 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.590306 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.590513 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.590632 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.590855 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.591056 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.591150 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.591373 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.591462 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.591707 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.591795 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.592031 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.592118 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.592356 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.592481 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.592670 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.592767 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.592977 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.593064 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.593299 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.593497 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.593591 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.593812 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.593907 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.594143 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.594267 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.594464 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.594550 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.594796 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.594884 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.595108 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.595196 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.595415 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.595500 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.595718 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.595805 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.596023 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.596110 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.596323 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.596410 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.596631 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.596852 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.596941 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.597197 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.597283 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.597511 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.597604 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.597815 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.597902 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.598113 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.598207 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.598428 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.598516 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.598757 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.598847 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.599088 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.599284 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.599395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.599624 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.599719 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.599934 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.600021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.600252 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.600341 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.600572 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.600671 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.600872 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.600959 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.601174 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.601267 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.601479 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.601566 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.601831 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.601921 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.602131 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.602225 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.602435 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.602522 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.602764 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.602851 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.603067 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.603158 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.603369 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.603456 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.603672 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.603758 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.603971 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.604065 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.604270 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.604393 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.604574 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.604662 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.604870 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.604968 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.605167 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.605263 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.605475 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.605561 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.605772 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.605986 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.606073 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.606158 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.606358 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.606447 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.606680 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.606789 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.606983 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.607070 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.607281 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.607367 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.607577 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.607675 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.607879 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.607975 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.608175 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.608262 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.608478 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.608567 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.608777 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.608864 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.609080 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.609224 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.609400 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.609494 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.609710 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.609797 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.610009 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.610094 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.610325 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.610413 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.610646 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.610738 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.611099 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.611204 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.611420 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.611508 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.611728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.611814 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.612045 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.612144 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.612351 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.612438 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.612655 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.612742 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.612998 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.613094 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.613296 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.613381 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.613617 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.613822 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.613910 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.614149 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.614244 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.614465 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.614552 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.614816 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.614915 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.615124 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.615211 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.615427 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.615520 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.615727 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.615823 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.616025 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.616112 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.616327 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.616413 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.616634 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.616747 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.616939 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.617040 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.617261 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.617352 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.617568 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.617655 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.617872 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.617959 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.618174 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.618259 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.618476 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.618571 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.618802 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.618889 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.619104 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.619192 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.619405 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.619500 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.619708 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.619794 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.620009 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.620096 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.620310 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.620398 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.620611 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.620698 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.620914 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.621012 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.621227 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.621314 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.621538 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.621622 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.621845 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.621941 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.622147 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.622241 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.622453 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.622778 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.622879 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.623089 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.623177 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.623388 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.623474 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.623689 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.623785 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.623996 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.624082 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.624317 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.624405 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.624622 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.624717 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.624921 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.625008 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.625248 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.625558 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.625643 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.625874 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.625960 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.626187 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.626273 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.626500 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.626608 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.626827 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.626914 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.627139 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.627224 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.627441 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.627541 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.627742 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.627828 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.628045 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.628132 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.628348 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.628446 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.628652 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.628738 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.628959 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.629046 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.629261 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.629349 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.629566 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.629661 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.629868 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.629955 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.630163 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.630256 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.630459 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.630548 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.630782 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.630869 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.631089 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.631178 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.631389 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.631477 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.631684 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.631769 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.631981 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.632069 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.632277 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.632362 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.632572 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.632666 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.633399 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.633495 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.633709 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.634050 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.634262 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.634363 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.634566 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.634710 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.634890 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.634977 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.635191 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.635358 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.635491 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.635580 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.635790 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.635878 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.636089 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.636189 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.636386 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.636479 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.636686 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.636773 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.636984 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.637071 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.637282 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.637369 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.637578 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.637671 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.637874 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.637963 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.638169 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.638257 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.638539 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.638663 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.638854 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.638949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.639160 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.639246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.639476 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.639571 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.639781 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.639866 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.640089 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.640175 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.640389 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.640483 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.640694 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.640788 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.641003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.641091 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.641310 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.641613 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.641699 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.641922 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.642008 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.642226 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.642323 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.642530 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.642642 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.642858 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.642958 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.643160 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.643255 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.643466 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.643561 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.643767 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.643854 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.644081 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.644168 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.644388 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.644474 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.644685 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.644810 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.644988 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.645075 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.645295 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.645382 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.645604 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.645701 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.645907 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.645992 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.646205 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.646299 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.646516 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.646637 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.646844 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.646938 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.647147 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.647234 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.647458 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.647554 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.647764 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.647851 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.648066 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.648154 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.648374 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.648464 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.648727 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.648817 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.649074 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.649168 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.649404 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.649492 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.649715 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.649802 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.650025 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.650110 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.650335 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.650424 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.650665 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.650781 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.650906 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 06:15:54.651037 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.651358 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.651695 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.652001 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.652336 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.652659 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.652966 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.653278 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.653578 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:15:54.653815 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 06:15:54.803931 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO dbcounter [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Registered counter for database nova_api Sep 18 06:15:54.805677 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO dbcounter [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Registered counter for database nova_api Sep 18 06:15:54.825018 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_db.sqlalchemy.engines [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:15:54.826660 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_db.sqlalchemy.engines [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105405) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:15:54.831782 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writer thread running {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:15:54.833043 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writer thread running {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:15:54.888850 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.context [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),63c4bc6f-f107-41b9-b2c1-7478fed24e9a(cell1) {{(pid=105406) load_cells /opt/stack/nova/nova/context.py:464}} Sep 18 06:15:54.889395 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.context [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),63c4bc6f-f107-41b9-b2c1-7478fed24e9a(cell1) {{(pid=105405) load_cells /opt/stack/nova/nova/context.py:464}} Sep 18 06:15:54.892661 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_concurrency.lockutils [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105406) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:15:54.893139 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_concurrency.lockutils [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105406) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:15:54.893232 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:15:54.893691 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:15:54.894733 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_concurrency.lockutils [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105406) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:15:54.895259 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:15:54.895470 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_concurrency.lockutils [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105406) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:15:54.895916 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_concurrency.lockutils [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105406) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:15:54.896011 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:15:54.896391 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:15:54.897909 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_concurrency.lockutils [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105406) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:15:54.898328 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:15:54.900506 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO dbcounter [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Registered counter for database nova_cell0 Sep 18 06:15:54.900870 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO dbcounter [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Registered counter for database nova_cell0 Sep 18 06:15:54.904536 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO dbcounter [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Registered counter for database nova_cell1 Sep 18 06:15:54.905037 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO dbcounter [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Registered counter for database nova_cell1 Sep 18 06:15:54.911622 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_db.sqlalchemy.engines [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:15:54.913185 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_db.sqlalchemy.engines [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:15:54.914840 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_db.sqlalchemy.engines [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105405) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:15:54.918202 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writer thread running {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:15:54.919043 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_db.sqlalchemy.engines [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105405) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:15:54.920581 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writer thread running {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:15:55.056304 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writer thread running {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:15:55.059822 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writer thread running {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:15:55.066138 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO dbcounter [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Registered counter for database nova_cell0 Sep 18 06:15:55.068177 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO dbcounter [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Registered counter for database nova_cell0 Sep 18 06:15:55.075078 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG oslo_db.sqlalchemy.engines [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:15:55.076617 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_db.sqlalchemy.engines [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105405) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:15:55.079853 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writer thread running {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:15:55.081175 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writer thread running {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:15:55.413935 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: WARNING keystonemiddleware.auth_token [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 06:15:55.422518 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: WARNING keystonemiddleware.auth_token [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 06:15:55.423782 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: WARNING keystonemiddleware.auth_token [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 06:15:55.431095 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: WARNING keystonemiddleware.auth_token [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 06:15:55.757927 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: WARNING keystonemiddleware.auth_token [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 06:15:55.764041 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: WARNING keystonemiddleware.auth_token [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 06:15:55.769243 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e2aea129c0 pid: 105405 (default app) Sep 18 06:15:55.769976 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: WARNING keystonemiddleware.auth_token [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 06:15:55.771355 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:15:55.772532 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] 10.222.0.20 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001797 Sep 18 06:15:55.772951 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 788 bytes} [Sun Sep 18 06:15:55 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:15:55.776428 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: WARNING keystonemiddleware.auth_token [None req-cdfa2781-6db6-4b25-8542-06cc0b5bcb78 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 06:15:55.781713 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e2aea129c0 pid: 105406 (default app) Sep 18 06:17:15.984649 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:15.985661 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001442 Sep 18 06:17:15.986053 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 2/2] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:16.000367 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writing DB stats nova_api:SELECT=3 {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:16.001451 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writing DB stats nova_cell1:SELECT=3 {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:16.003284 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writing DB stats nova_cell0:SELECT=3 {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:16.005447 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:16.864059 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-2708bd11-a6b9-4eae-98c4-927cbab8ad03 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:16.896999 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-2708bd11-a6b9-4eae-98c4-927cbab8ad03 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.903222 Sep 18 06:17:16.897943 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 1/3] 10.222.0.20 () {60 vars in 1292 bytes} [Sun Sep 18 06:17:15 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 906 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:17:19.519053 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:19.520183 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-9d85ebe8-e4b1-4d9d-889d-c4300d0692f5 None None] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001530 Sep 18 06:17:19.520824 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 3/4] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:19 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:19.971141 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-4042d534-4ded-4522-b003-52eaeac6e18a admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 256, "id": "c1", "disk": 1, "name": "cirros256", "swap": 0, "os-flavor-access:is_public": true, "vcpus": 1, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:20.020155 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-4042d534-4ded-4522-b003-52eaeac6e18a admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.493218 Sep 18 06:17:20.020721 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 2/5] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:19 2022] POST /compute/v2.1/flavors => generated 407 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 18 06:17:20.035914 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_api:SELECT=3 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:20.037967 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_cell0:SELECT=3 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:20.038738 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_cell1:SELECT=3 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:20.039816 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:20.243485 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-0c1a583a-11e5-4b64-ab33-0832278caa28 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:20.311864 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-0c1a583a-11e5-4b64-ab33-0832278caa28 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.281453 Sep 18 06:17:20.314685 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 4/6] 10.222.0.20 () {64 vars in 1336 bytes} [Sun Sep 18 06:17:20 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:17:22.928326 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-4042d534-4ded-4522-b003-52eaeac6e18a admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:22.929363 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-4042d534-4ded-4522-b003-52eaeac6e18a admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001497 Sep 18 06:17:22.929776 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 3/7] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:17:23.585371 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-87f89f1d-d60a-4f8c-9ddf-4718b73cd9eb admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "id": "d1", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 5, "rxtx_factor": 1.0, "ram": 512, "os-flavor-access:is_public": true, "name": "ds512M"}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:23.623214 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-87f89f1d-d60a-4f8c-9ddf-4718b73cd9eb admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.686325 Sep 18 06:17:23.623967 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 5/8] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:22 2022] POST /compute/v2.1/flavors => generated 404 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:17:23.636834 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-b54d6ef7-7a55-44c9-9ebc-25a46e625df3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:23.679745 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-b54d6ef7-7a55-44c9-9ebc-25a46e625df3 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.047126 Sep 18 06:17:23.680489 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 4/9] 10.222.0.20 () {64 vars in 1336 bytes} [Sun Sep 18 06:17:23 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:17:26.305962 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-87f89f1d-d60a-4f8c-9ddf-4718b73cd9eb admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:26.307015 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-87f89f1d-d60a-4f8c-9ddf-4718b73cd9eb admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001443 Sep 18 06:17:26.307427 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 6/10] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:26.727535 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-b7c8229e-cdb0-4b63-8711-f2a52c7579b1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds1G", "os-flavor-access:is_public": true, "disk": 10, "swap": 0, "id": "d2", "rxtx_factor": 1.0, "vcpus": 1, "ram": 1024, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:26.748984 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-b7c8229e-cdb0-4b63-8711-f2a52c7579b1 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.435648 Sep 18 06:17:26.749542 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 5/11] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:26 2022] POST /compute/v2.1/flavors => generated 404 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:17:26.764029 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-9f4162ab-fd50-453f-a11d-c4b0b251ed64 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:26.792892 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-9f4162ab-fd50-453f-a11d-c4b0b251ed64 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034094 Sep 18 06:17:26.793456 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 7/12] 10.222.0.20 () {64 vars in 1336 bytes} [Sun Sep 18 06:17:26 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:17:29.402764 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-b7c8229e-cdb0-4b63-8711-f2a52c7579b1 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:29.403781 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-b7c8229e-cdb0-4b63-8711-f2a52c7579b1 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001496 Sep 18 06:17:29.404603 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 6/13] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:29 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:29.830358 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-238564b9-d24f-4c89-9182-cc4aa9d1c131 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "vcpus": 2, "swap": 0, "name": "ds2G", "disk": 10, "id": "d3", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:29.854003 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-238564b9-d24f-4c89-9182-cc4aa9d1c131 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.442556 Sep 18 06:17:29.855177 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 8/14] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:29 2022] POST /compute/v2.1/flavors => generated 404 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:17:29.868271 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-2bcbd2b1-8a89-44ad-a0de-c0616e7398e6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:29.896716 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-2bcbd2b1-8a89-44ad-a0de-c0616e7398e6 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032483 Sep 18 06:17:29.898070 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 7/15] 10.222.0.20 () {64 vars in 1336 bytes} [Sun Sep 18 06:17:29 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:17:32.485902 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-238564b9-d24f-4c89-9182-cc4aa9d1c131 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:32.486957 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-238564b9-d24f-4c89-9182-cc4aa9d1c131 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001515 Sep 18 06:17:32.487361 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 9/16] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:32 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:32.910837 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-94212021-fe4a-4a65-a0f6-15e8ae72404c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds4G", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "id": "d4", "ram": 4096, "vcpus": 4}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:32.930647 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-94212021-fe4a-4a65-a0f6-15e8ae72404c admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.437244 Sep 18 06:17:32.931260 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 8/17] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:32 2022] POST /compute/v2.1/flavors => generated 404 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:17:32.944398 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-6e33dcf3-53fd-4759-a497-387daf5fa3b6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:32.970101 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-6e33dcf3-53fd-4759-a497-387daf5fa3b6 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029013 Sep 18 06:17:32.970763 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 10/18] 10.222.0.20 () {64 vars in 1336 bytes} [Sun Sep 18 06:17:32 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:17:35.937673 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-94212021-fe4a-4a65-a0f6-15e8ae72404c admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:35.939411 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-94212021-fe4a-4a65-a0f6-15e8ae72404c admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002173 Sep 18 06:17:35.940057 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 9/19] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:36.399879 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-a35dc07f-8e7a-4ef1-83fb-effc7b6010cf admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:36.431957 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-a35dc07f-8e7a-4ef1-83fb-effc7b6010cf admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.481734 Sep 18 06:17:36.432780 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 11/20] 10.222.0.20 () {60 vars in 1292 bytes} [Sun Sep 18 06:17:35 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:17:39.395470 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-94212021-fe4a-4a65-a0f6-15e8ae72404c admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:39.396398 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-94212021-fe4a-4a65-a0f6-15e8ae72404c admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001478 Sep 18 06:17:39.396770 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 10/21] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:39.808215 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-570852a9-2251-49f1-b4f1-7a04b6e2b8ab admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "vcpus": 1, "disk": 1, "ram": 512, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "swap": 0, "name": "m1.tiny", "id": "1"}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:39.832695 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-570852a9-2251-49f1-b4f1-7a04b6e2b8ab admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.429983 Sep 18 06:17:39.833319 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 12/22] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:39 2022] POST /compute/v2.1/flavors => generated 402 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 18 06:17:39.845973 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-f92d425d-5ffe-4de6-baaa-28b43e501f0a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:39.873464 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-f92d425d-5ffe-4de6-baaa-28b43e501f0a admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031747 Sep 18 06:17:39.874038 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 11/23] 10.222.0.20 () {64 vars in 1333 bytes} [Sun Sep 18 06:17:39 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:17:42.502975 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-570852a9-2251-49f1-b4f1-7a04b6e2b8ab admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:42.503983 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-570852a9-2251-49f1-b4f1-7a04b6e2b8ab admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001474 Sep 18 06:17:42.504383 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 13/24] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:42.876313 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-03b8fa9b-ea8c-41b9-9276-d913895a5b92 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "id": "2", "name": "m1.small", "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 20}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:42.896096 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-03b8fa9b-ea8c-41b9-9276-d913895a5b92 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.385202 Sep 18 06:17:42.896596 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 12/25] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:42 2022] POST /compute/v2.1/flavors => generated 405 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:17:42.909258 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-41dbb2fe-b3cf-4172-b637-dfd890536e0a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:42.932784 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-41dbb2fe-b3cf-4172-b637-dfd890536e0a admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027709 Sep 18 06:17:42.933403 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 14/26] 10.222.0.20 () {64 vars in 1333 bytes} [Sun Sep 18 06:17:42 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:17:45.553094 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-03b8fa9b-ea8c-41b9-9276-d913895a5b92 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:45.554024 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-03b8fa9b-ea8c-41b9-9276-d913895a5b92 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001460 Sep 18 06:17:45.554426 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 13/27] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:46.006939 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-c1d5ac84-2250-477d-b9e9-a19c20a42a9f admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "3", "vcpus": 2, "disk": 40, "name": "m1.medium", "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "swap": 0}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:46.027145 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-c1d5ac84-2250-477d-b9e9-a19c20a42a9f admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.466592 Sep 18 06:17:46.027976 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 15/28] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:45 2022] POST /compute/v2.1/flavors => generated 406 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 18 06:17:46.040596 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-8866d511-9600-4b93-94af-67501c80ada2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:46.047088 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:46.074657 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-8866d511-9600-4b93-94af-67501c80ada2 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.038123 Sep 18 06:17:46.083829 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 14/29] 10.222.0.20 () {64 vars in 1333 bytes} [Sun Sep 18 06:17:46 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:17:48.759847 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-c1d5ac84-2250-477d-b9e9-a19c20a42a9f admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:48.760869 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-c1d5ac84-2250-477d-b9e9-a19c20a42a9f admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Sep 18 06:17:48.761280 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 16/30] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:49.206686 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-7049e11c-8e45-46a4-9226-1cc37be8a374 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "4", "name": "m1.large", "swap": 0, "os-flavor-access:is_public": true, "disk": 80, "vcpus": 4, "rxtx_factor": 1.0, "ram": 8192, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:49.225884 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-7049e11c-8e45-46a4-9226-1cc37be8a374 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.458104 Sep 18 06:17:49.226450 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 15/31] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:48 2022] POST /compute/v2.1/flavors => generated 405 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:17:49.239080 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-ab0f8280-82db-49e9-a2ab-a9bc32392815 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:49.269464 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-ab0f8280-82db-49e9-a2ab-a9bc32392815 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034643 Sep 18 06:17:49.270018 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 17/32] 10.222.0.20 () {64 vars in 1333 bytes} [Sun Sep 18 06:17:49 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:17:51.878630 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-7049e11c-8e45-46a4-9226-1cc37be8a374 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:17:51.879567 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-7049e11c-8e45-46a4-9226-1cc37be8a374 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001483 Sep 18 06:17:51.879960 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 16/33] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:17:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:17:52.359068 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-388d93fa-42e8-41d0-9f63-a21ce9883fa1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 8, "swap": 0, "rxtx_factor": 1.0, "name": "m1.xlarge", "id": "5", "os-flavor-access:is_public": true, "disk": 160}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:52.367454 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:52.411041 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-388d93fa-42e8-41d0-9f63-a21ce9883fa1 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.524482 Sep 18 06:17:52.411536 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 18/34] 10.222.0.20 () {64 vars in 1283 bytes} [Sun Sep 18 06:17:51 2022] POST /compute/v2.1/flavors => generated 408 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:17:52.423591 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-1aac8dd7-4886-445f-8efd-a38760f70544 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:17:52.448874 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-1aac8dd7-4886-445f-8efd-a38760f70544 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028992 Sep 18 06:17:52.449833 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 17/35] 10.222.0.20 () {64 vars in 1333 bytes} [Sun Sep 18 06:17:52 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:18:08.999130 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-388d93fa-42e8-41d0-9f63-a21ce9883fa1 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:09.000074 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-388d93fa-42e8-41d0-9f63-a21ce9883fa1 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001507 Sep 18 06:18:09.001008 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 19/36] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:18:08 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:18:09.011563 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:09.388251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-2b43684e-6b2b-4446-a0eb-77d990b2acf2 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:09.406344 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-2b43684e-6b2b-4446-a0eb-77d990b2acf2 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.399230 Sep 18 06:18:09.406931 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 18/37] 10.222.0.20 () {60 vars in 1292 bytes} [Sun Sep 18 06:18:09 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:18:10.724107 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-388d93fa-42e8-41d0-9f63-a21ce9883fa1 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:10.725176 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-388d93fa-42e8-41d0-9f63-a21ce9883fa1 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 Sep 18 06:18:10.725528 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 20/38] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:18:10 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:18:11.237761 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-895438f9-47d6-4364-8ace-8e737acc1b09 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:11.243150 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.wsgi [None req-895438f9-47d6-4364-8ace-8e737acc1b09 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-895438f9-47d6-4364-8ace-8e737acc1b09 admin admin] Request method failure captured: Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Accept: application/json Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Accept-Encoding: gzip, deflate Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Connection: keep-alive Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Content-Length: 0 Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Host: 10.222.0.20 Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Openstack-Api-Version: compute 2.92 Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Openstack-System-Scope: None Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: User-Agent: python-novaclient Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Auth-Token: gAAAAABjJrgiWePn5zrneZnEJvd450O3g7LyUFWv7FqfNmCCXSnj1EcJ54IGpPUwHqC0ZEyBNLaDG95zfAFzRRPqf7315Sxb4tNsEPhHl-qxtJ9CUgXostgfFqT2PqqxLsj33uzpW8PG80k5OMW724SL1uXcc04N4TJt_yyU3gt4WnqWDfTWX_c Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Domain-Id: None Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Domain-Name: None Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Identity-Status: Confirmed Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Is-Admin-Project: True Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Domain-Id: default Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Domain-Name: Default Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Name: admin Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Role: reader,admin,member Sep 18 06:18:11.246395 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Roles: reader,admin,member Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant: admin Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant-Name: admin Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User: admin Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Domain-Id: default Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Domain-Name: Default Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Id: 949debb5f3604102a418027d122cd476 Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Name: admin Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: method: > Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: exception: Flavor 42 could not be found. Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: action_args: {'id': '42'} Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:18:11.248021 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:11.248996 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.250300 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-895438f9-47d6-4364-8ace-8e737acc1b09 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105406) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:18:11.252086 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-895438f9-47d6-4364-8ace-8e737acc1b09 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.387165 Sep 18 06:18:11.252086 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 19/39] 10.222.0.20 () {60 vars in 1190 bytes} [Sun Sep 18 06:18:10 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 388 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:11.258277 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-38aac790-a4b2-434b-ad00-ada95cf24c5b admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:11.281110 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-38aac790-a4b2-434b-ad00-ada95cf24c5b admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026531 Sep 18 06:18:11.281577 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 21/40] 10.222.0.20 () {60 vars in 1210 bytes} [Sun Sep 18 06:18:11 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:18:11.291905 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-2fdc643e-c79a-44ca-b6da-b40d3c661fa5 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:11.313727 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-2fdc643e-c79a-44ca-b6da-b40d3c661fa5 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.025740 Sep 18 06:18:11.314269 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 20/41] 10.222.0.20 () {60 vars in 1210 bytes} [Sun Sep 18 06:18:11 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:18:11.323412 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-b30877e0-1b9c-4bc5-ac96-4d0fa4dbff53 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:11.328504 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.wsgi [None req-b30877e0-1b9c-4bc5-ac96-4d0fa4dbff53 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-b30877e0-1b9c-4bc5-ac96-4d0fa4dbff53 admin admin] Request method failure captured: Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Accept: application/json Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Accept-Encoding: gzip, deflate Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Connection: keep-alive Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Content-Length: 0 Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Host: 10.222.0.20 Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Openstack-Api-Version: compute 2.92 Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Openstack-System-Scope: None Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: User-Agent: python-novaclient Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Auth-Token: gAAAAABjJrgiWePn5zrneZnEJvd450O3g7LyUFWv7FqfNmCCXSnj1EcJ54IGpPUwHqC0ZEyBNLaDG95zfAFzRRPqf7315Sxb4tNsEPhHl-qxtJ9CUgXostgfFqT2PqqxLsj33uzpW8PG80k5OMW724SL1uXcc04N4TJt_yyU3gt4WnqWDfTWX_c Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Domain-Id: None Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Domain-Name: None Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Identity-Status: Confirmed Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Is-Admin-Project: True Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Domain-Id: default Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Domain-Name: Default Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Name: admin Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Role: reader,admin,member Sep 18 06:18:11.332426 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Roles: reader,admin,member Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant: admin Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant-Name: admin Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User: admin Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Domain-Id: default Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Domain-Name: Default Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Id: 949debb5f3604102a418027d122cd476 Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Name: admin Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: method: > Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: exception: Flavor 42 could not be found. Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: action_args: {'id': '42'} Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:18:11.333883 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:11.334849 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:11.336084 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-b30877e0-1b9c-4bc5-ac96-4d0fa4dbff53 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105405) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:18:11.337284 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-b30877e0-1b9c-4bc5-ac96-4d0fa4dbff53 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014072 Sep 18 06:18:11.337284 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 22/42] 10.222.0.20 () {60 vars in 1190 bytes} [Sun Sep 18 06:18:11 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:12.539063 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-2fdc643e-c79a-44ca-b6da-b40d3c661fa5 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:12.540081 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-2fdc643e-c79a-44ca-b6da-b40d3c661fa5 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001545 Sep 18 06:18:12.540481 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 21/43] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:18:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:18:13.055781 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-26920bb5-819e-402e-83d9-2adea2880fc7 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:13.060999 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.wsgi [None req-26920bb5-819e-402e-83d9-2adea2880fc7 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-26920bb5-819e-402e-83d9-2adea2880fc7 admin admin] Request method failure captured: Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Accept: application/json Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Accept-Encoding: gzip, deflate Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Connection: keep-alive Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Content-Length: 0 Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Host: 10.222.0.20 Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Openstack-Api-Version: compute 2.92 Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Openstack-System-Scope: None Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: User-Agent: python-novaclient Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Auth-Token: gAAAAABjJrgkagcQXi-J_hVH4WmhkML6Hn5f5A0O99mqZHxhOPFQYrwHs_bILmAELu2jnZEo2Y5zVDjLbNc1mUtM0lXjcSvm9S_grTBkZ84x50MfEPBOIE6Ko_Sr7ARcWn4A4aQ-s8urqZ12ilNN2U1nbqMq4oNPAzUB5dJR6hkz4OeN9gb1IJk Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Domain-Id: None Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Domain-Name: None Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Identity-Status: Confirmed Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Is-Admin-Project: True Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Domain-Id: default Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Domain-Name: Default Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Name: admin Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Role: reader,admin,member Sep 18 06:18:13.062502 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Roles: reader,admin,member Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant: admin Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant-Name: admin Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User: admin Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Domain-Id: default Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Domain-Name: Default Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Id: 949debb5f3604102a418027d122cd476 Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Name: admin Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: method: > Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: exception: Flavor 84 could not be found. Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: action_args: {'id': '84'} Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:18:13.064728 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:13.065550 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.066949 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-26920bb5-819e-402e-83d9-2adea2880fc7 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105405) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:18:13.068171 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-26920bb5-819e-402e-83d9-2adea2880fc7 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.387909 Sep 18 06:18:13.068171 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 23/44] 10.222.0.20 () {60 vars in 1190 bytes} [Sun Sep 18 06:18:12 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 389 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:13.074155 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-91cbc36b-f4a6-43b5-8f7c-254ec3335d40 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:13.088759 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-91cbc36b-f4a6-43b5-8f7c-254ec3335d40 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018180 Sep 18 06:18:13.089282 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 22/45] 10.222.0.20 () {60 vars in 1210 bytes} [Sun Sep 18 06:18:13 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:18:13.098269 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-efa03af6-ed49-4fbe-86ef-ba1ece0f6c72 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:13.113854 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-efa03af6-ed49-4fbe-86ef-ba1ece0f6c72 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019007 Sep 18 06:18:13.114383 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 24/46] 10.222.0.20 () {60 vars in 1210 bytes} [Sun Sep 18 06:18:13 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:18:13.123260 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-70994f46-6bb9-457c-b0f8-ff6df088cc4f admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:13.128766 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.wsgi [None req-70994f46-6bb9-457c-b0f8-ff6df088cc4f admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-70994f46-6bb9-457c-b0f8-ff6df088cc4f admin admin] Request method failure captured: Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Accept: application/json Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Accept-Encoding: gzip, deflate Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Connection: keep-alive Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Content-Length: 0 Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Host: 10.222.0.20 Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Openstack-Api-Version: compute 2.92 Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Openstack-System-Scope: None Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: User-Agent: python-novaclient Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Auth-Token: gAAAAABjJrgkagcQXi-J_hVH4WmhkML6Hn5f5A0O99mqZHxhOPFQYrwHs_bILmAELu2jnZEo2Y5zVDjLbNc1mUtM0lXjcSvm9S_grTBkZ84x50MfEPBOIE6Ko_Sr7ARcWn4A4aQ-s8urqZ12ilNN2U1nbqMq4oNPAzUB5dJR6hkz4OeN9gb1IJk Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Domain-Id: None Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Domain-Name: None Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Identity-Status: Confirmed Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Is-Admin-Project: True Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Domain-Id: default Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Domain-Name: Default Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Name: admin Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Role: reader,admin,member Sep 18 06:18:13.130246 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Roles: reader,admin,member Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant: admin Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant-Name: admin Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User: admin Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Domain-Id: default Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Domain-Name: Default Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Id: 949debb5f3604102a418027d122cd476 Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Name: admin Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: method: > Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: exception: Flavor 84 could not be found. Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: action_args: {'id': '84'} Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:18:13.132003 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:13.133083 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:13.134626 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-70994f46-6bb9-457c-b0f8-ff6df088cc4f admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105406) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:18:13.135903 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-70994f46-6bb9-457c-b0f8-ff6df088cc4f admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011763 Sep 18 06:18:13.135903 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 23/47] 10.222.0.20 () {60 vars in 1190 bytes} [Sun Sep 18 06:18:13 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:14.340798 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-efa03af6-ed49-4fbe-86ef-ba1ece0f6c72 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:14.341797 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-efa03af6-ed49-4fbe-86ef-ba1ece0f6c72 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001427 Sep 18 06:18:14.342190 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 25/48] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:18:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:18:14.863941 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-05141345-d0e1-4108-8b30-5b8d81b40e1b admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:14.869707 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.wsgi [None req-05141345-d0e1-4108-8b30-5b8d81b40e1b admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-05141345-d0e1-4108-8b30-5b8d81b40e1b admin admin] Request method failure captured: Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Accept: application/json Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Accept-Encoding: gzip, deflate Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Connection: keep-alive Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Content-Length: 0 Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Host: 10.222.0.20 Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Openstack-Api-Version: compute 2.92 Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Openstack-System-Scope: None Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: User-Agent: python-novaclient Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Auth-Token: gAAAAABjJrgmZjDEGCjEwcKO6avxjAWddj94Naim6RMTowrYZINH0ehFQmvli-9NAxYhbhBAOMd2nV-cx0zJ9LaOeGN4-L4VKQi7sFNFGosxCQZj-1RrzHIesmB431kQZ_r33oCNlYlna6G6G5sv26jvYyCrW9Y5Uxx7NgmTQRS_lHa6OBYan8Y Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Domain-Id: None Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Domain-Name: None Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Identity-Status: Confirmed Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Is-Admin-Project: True Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Domain-Id: default Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Domain-Name: Default Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Project-Name: admin Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Role: reader,admin,member Sep 18 06:18:14.871222 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Roles: reader,admin,member Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant: admin Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-Tenant-Name: admin Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User: admin Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Domain-Id: default Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Domain-Name: Default Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Id: 949debb5f3604102a418027d122cd476 Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: X-User-Name: admin Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: method: > Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: exception: Flavor 451 could not be found. Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: action_args: {'id': '451'} Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:18:14.872825 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:14.873859 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.875251 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-05141345-d0e1-4108-8b30-5b8d81b40e1b admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105406) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:18:14.876510 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-05141345-d0e1-4108-8b30-5b8d81b40e1b admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.393073 Sep 18 06:18:14.876510 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 24/49] 10.222.0.20 () {60 vars in 1193 bytes} [Sun Sep 18 06:18:14 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 394 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:14.882764 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-0259208b-3544-4c6f-b006-a629797a9f0d admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:14.897341 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-0259208b-3544-4c6f-b006-a629797a9f0d admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018425 Sep 18 06:18:14.897857 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 26/50] 10.222.0.20 () {60 vars in 1210 bytes} [Sun Sep 18 06:18:14 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:18:14.907670 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:14.922124 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018490 Sep 18 06:18:14.923346 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 25/51] 10.222.0.20 () {60 vars in 1210 bytes} [Sun Sep 18 06:18:14 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:18:14.932333 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-f5a1bf0c-538c-4eb2-94aa-dd607a52ad7e admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:14.937085 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.wsgi [None req-f5a1bf0c-538c-4eb2-94aa-dd607a52ad7e admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-f5a1bf0c-538c-4eb2-94aa-dd607a52ad7e admin admin] Request method failure captured: Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Accept: application/json Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Accept-Encoding: gzip, deflate Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Connection: keep-alive Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Content-Length: 0 Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Host: 10.222.0.20 Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Openstack-Api-Version: compute 2.92 Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Openstack-System-Scope: None Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: User-Agent: python-novaclient Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Auth-Token: gAAAAABjJrgmZjDEGCjEwcKO6avxjAWddj94Naim6RMTowrYZINH0ehFQmvli-9NAxYhbhBAOMd2nV-cx0zJ9LaOeGN4-L4VKQi7sFNFGosxCQZj-1RrzHIesmB431kQZ_r33oCNlYlna6G6G5sv26jvYyCrW9Y5Uxx7NgmTQRS_lHa6OBYan8Y Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Domain-Id: None Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Domain-Name: None Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Identity-Status: Confirmed Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Is-Admin-Project: True Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Domain-Id: default Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Domain-Name: Default Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Project-Name: admin Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Role: reader,admin,member Sep 18 06:18:14.938697 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Roles: reader,admin,member Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/39ab4a771cc044ca930df75a3565efaa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant: admin Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant-Id: 39ab4a771cc044ca930df75a3565efaa Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-Tenant-Name: admin Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User: admin Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Domain-Id: default Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Domain-Name: Default Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Id: 949debb5f3604102a418027d122cd476 Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: X-User-Name: admin Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: method: > Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: exception: Flavor 451 could not be found. Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: action_args: {'id': '451'} Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:18:14.940206 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:14.941377 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: ERROR nova.api.openstack.wsgi Sep 18 06:18:14.942629 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-f5a1bf0c-538c-4eb2-94aa-dd607a52ad7e admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105405) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:18:14.944012 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-f5a1bf0c-538c-4eb2-94aa-dd607a52ad7e admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011192 Sep 18 06:18:14.944012 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 27/52] 10.222.0.20 () {60 vars in 1193 bytes} [Sun Sep 18 06:18:14 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:16.158828 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:16.160059 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001986 Sep 18 06:18:16.160420 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:18:16 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:18:16.675013 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-456b5b42-6a60-448e-bdc0-953559073437 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:18:16.695050 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-456b5b42-6a60-448e-bdc0-953559073437 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.398776 Sep 18 06:18:16.695605 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 28/54] 10.222.0.20 () {64 vars in 1235 bytes} [Sun Sep 18 06:18:16 2022] POST /compute/v2.1/flavors => generated 404 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:18:18.026040 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:18.027134 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001517 Sep 18 06:18:18.027573 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 27/55] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:18:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:18:18.544687 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-a550000f-f355-471a-a443-d14f9e7b46a9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:18:18.563751 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-a550000f-f355-471a-a443-d14f9e7b46a9 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.400305 Sep 18 06:18:18.564343 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 29/56] 10.222.0.20 () {64 vars in 1235 bytes} [Sun Sep 18 06:18:18 2022] POST /compute/v2.1/flavors => generated 405 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 18 06:18:19.770042 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:19.771077 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001451 Sep 18 06:18:19.771462 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 28/57] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:18:19 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:18:20.277421 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-369d4358-e344-4763-951f-a8989c40c80c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:18:20.295882 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-369d4358-e344-4763-951f-a8989c40c80c admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.387616 Sep 18 06:18:20.296416 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 30/58] 10.222.0.20 () {64 vars in 1235 bytes} [Sun Sep 18 06:18:19 2022] POST /compute/v2.1/flavors => generated 407 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:18:21.490694 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:21.491692 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Sep 18 06:18:21.492077 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 29/59] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:18:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:18:22.006237 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-cf4f6175-7b37-4fcb-9f16-e9d780c901e6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:18:22.025241 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-cf4f6175-7b37-4fcb-9f16-e9d780c901e6 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.395951 Sep 18 06:18:22.025789 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 31/60] 10.222.0.20 () {64 vars in 1235 bytes} [Sun Sep 18 06:18:21 2022] POST /compute/v2.1/flavors => generated 409 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:18:23.229123 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:23.230125 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001420 Sep 18 06:18:23.230512 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 30/61] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:18:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:18:23.739045 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-b0c1ed6a-7a7f-416c-b756-4bf3f60ff218 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:18:23.759587 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-b0c1ed6a-7a7f-416c-b756-4bf3f60ff218 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.394322 Sep 18 06:18:23.760126 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 32/62] 10.222.0.20 () {60 vars in 1202 bytes} [Sun Sep 18 06:18:23 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:19:20.026217 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:19:20.027266 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-e4389610-7f87-4240-90b5-bde0dc0e4cc4 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001482 Sep 18 06:19:20.027664 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 31/63] 10.222.0.20 () {54 vars in 931 bytes} [Sun Sep 18 06:19:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:19:20.038481 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:20.418897 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-a1090011-809c-4356-b23e-48b39e02c931 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:19:20.439437 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-a1090011-809c-4356-b23e-48b39e02c931 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.404781 Sep 18 06:19:20.439976 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 33/64] 10.222.0.20 () {60 vars in 1292 bytes} [Sun Sep 18 06:19:20 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:19:20.461108 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writing DB stats nova_api:SELECT=14 {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:20.465023 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-1e14c5f9-44d3-4ece-b7fd-fa9c4a5862af admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:19:20.472286 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-1e14c5f9-44d3-4ece-b7fd-fa9c4a5862af admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012576 Sep 18 06:19:20.472809 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 32/65] 10.222.0.20 () {60 vars in 1283 bytes} [Sun Sep 18 06:19:20 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:19:20.484600 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-12dd8d0c-8fe7-450e-9f5e-3f96c09a0daf admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:19:20.492393 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-12dd8d0c-8fe7-450e-9f5e-3f96c09a0daf admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011506 Sep 18 06:19:20.492916 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 34/66] 10.222.0.20 () {60 vars in 1286 bytes} [Sun Sep 18 06:19:20 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:19:20.505244 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:19:20.512843 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011789 Sep 18 06:19:20.513351 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 33/67] 10.222.0.20 () {60 vars in 1286 bytes} [Sun Sep 18 06:19:20 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:19:20.524469 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-e903aabd-12ab-4432-81ab-cb26651e1182 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:19:20.531567 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-e903aabd-12ab-4432-81ab-cb26651e1182 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010569 Sep 18 06:19:20.532070 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 35/68] 10.222.0.20 () {60 vars in 1283 bytes} [Sun Sep 18 06:19:20 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:22:19.860142 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:22:19.861215 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001431 Sep 18 06:22:19.861614 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 34/69] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:22:19 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:22:20.003062 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_api:SELECT=6 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:20.166887 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-4253039b-40fe-4a30-91df-fad7bc77b387 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:22:20.173722 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-4253039b-40fe-4a30-91df-fad7bc77b387 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:22:20.174350 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-4253039b-40fe-4a30-91df-fad7bc77b387 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:22:20.174893 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-4253039b-40fe-4a30-91df-fad7bc77b387 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:22:20.175741 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-4253039b-40fe-4a30-91df-fad7bc77b387 admin admin] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:22:20.176247 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-4253039b-40fe-4a30-91df-fad7bc77b387 admin admin] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:22:20.176768 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-4253039b-40fe-4a30-91df-fad7bc77b387 admin admin] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:22:20.207796 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-4253039b-40fe-4a30-91df-fad7bc77b387 admin admin] 10.222.0.20 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.205999 Sep 18 06:22:20.208352 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 36/70] 10.222.0.20 () {60 vars in 1193 bytes} [Sun Sep 18 06:22:20 2022] GET /compute/v2.1/os-services => generated 791 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:22:21.272599 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:22:21.273592 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001408 Sep 18 06:22:21.273979 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 35/71] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:22:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:22:21.475245 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-a5a2ae4d-e55b-41d8-9aaf-66c0f42372ef admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:22:21.478682 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-a5a2ae4d-e55b-41d8-9aaf-66c0f42372ef admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:22:21.479160 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-a5a2ae4d-e55b-41d8-9aaf-66c0f42372ef admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:22:21.479585 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-a5a2ae4d-e55b-41d8-9aaf-66c0f42372ef admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:22:21.480300 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-a5a2ae4d-e55b-41d8-9aaf-66c0f42372ef admin admin] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:22:21.480748 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-a5a2ae4d-e55b-41d8-9aaf-66c0f42372ef admin admin] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:22:21.481162 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-a5a2ae4d-e55b-41d8-9aaf-66c0f42372ef admin admin] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:22:21.500175 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-a5a2ae4d-e55b-41d8-9aaf-66c0f42372ef admin admin] 10.222.0.20 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.080917 Sep 18 06:22:21.500725 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 37/72] 10.222.0.20 () {60 vars in 1193 bytes} [Sun Sep 18 06:22:21 2022] GET /compute/v2.1/os-services => generated 791 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:22:32.980472 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:22:32.981469 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001436 Sep 18 06:22:32.981863 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 36/73] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:22:32 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:22:33.124586 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_cell0:SELECT=4 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:33.125292 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_cell1:SELECT=4 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:33.125963 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG dbcounter [-] [105405] Writing DB stats nova_api:SELECT=8 {{(pid=105405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:33.175046 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-03011ffe-0f81-4e18-a5e7-550896ec4f37 admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:22:33.178382 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-03011ffe-0f81-4e18-a5e7-550896ec4f37 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:22:33.178887 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-03011ffe-0f81-4e18-a5e7-550896ec4f37 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:22:33.179312 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-03011ffe-0f81-4e18-a5e7-550896ec4f37 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:22:33.180021 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-03011ffe-0f81-4e18-a5e7-550896ec4f37 admin admin] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:22:33.180465 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-03011ffe-0f81-4e18-a5e7-550896ec4f37 admin admin] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:22:33.180877 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-03011ffe-0f81-4e18-a5e7-550896ec4f37 admin admin] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:22:33.205813 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-03011ffe-0f81-4e18-a5e7-550896ec4f37 admin admin] 10.222.0.20 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.086091 Sep 18 06:22:33.206273 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 38/74] 10.222.0.20 () {60 vars in 1193 bytes} [Sun Sep 18 06:22:33 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:22:34.271950 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG nova.api.openstack.wsgi [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] Calling method '>' {{(pid=105406) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:22:34.272949 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: INFO nova.api.openstack.requestlog [None req-51afc79b-61fb-49ac-8f99-19352e776972 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001404 Sep 18 06:22:34.273336 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: [pid: 105406|app: 0|req: 37/75] 10.222.0.20 () {56 vars in 1074 bytes} [Sun Sep 18 06:22:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:22:34.462201 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG nova.api.openstack.wsgi [None req-9387d457-8f10-4666-9ab4-ddef3254932d admin admin] Calling method '>' {{(pid=105405) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:22:34.465920 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9387d457-8f10-4666-9ab4-ddef3254932d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:22:34.466388 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9387d457-8f10-4666-9ab4-ddef3254932d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:22:34.466837 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9387d457-8f10-4666-9ab4-ddef3254932d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:22:34.467561 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9387d457-8f10-4666-9ab4-ddef3254932d admin admin] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:22:34.467997 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9387d457-8f10-4666-9ab4-ddef3254932d admin admin] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:22:34.468402 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: DEBUG oslo_concurrency.lockutils [None req-9387d457-8f10-4666-9ab4-ddef3254932d admin admin] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105405) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:22:34.489490 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: INFO nova.api.openstack.requestlog [None req-9387d457-8f10-4666-9ab4-ddef3254932d admin admin] 10.222.0.20 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.078427 Sep 18 06:22:34.489978 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: [pid: 105405|app: 0|req: 39/76] 10.222.0.20 () {60 vars in 1193 bytes} [Sun Sep 18 06:22:34 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:23:30.812651 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: Sun Sep 18 06:23:30 2022 - graceful shutdown triggered... Sep 18 06:23:30.812776 dv-cinder-855359-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 18 06:23:30.814174 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: Gracefully killing worker 2 (pid: 105406)... Sep 18 06:23:30.814628 dv-cinder-855359-3-iscsi devstack@n-api.service[105405]: Gracefully killing worker 1 (pid: 105405)... Sep 18 06:23:30.940014 dv-cinder-855359-3-iscsi devstack@n-api.service[105406]: DEBUG dbcounter [-] [105406] Writing DB stats nova_api:SELECT=4 {{(pid=105406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:23:31.815354 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: worker 1 buried after 1 seconds Sep 18 06:23:31.815354 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: worker 2 buried after 1 seconds Sep 18 06:23:31.815354 dv-cinder-855359-3-iscsi devstack@n-api.service[105404]: goodbye to uWSGI. Sep 18 06:23:31.816865 dv-cinder-855359-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 18 06:23:31.817422 dv-cinder-855359-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 18 06:23:31.820245 dv-cinder-855359-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 18 06:23:31.829220 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 18 06:23:31.832054 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 18 06:23:31 2022] *** Sep 18 06:23:31.832220 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 18 06:23:31.832354 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 18 06:23:31.832471 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: nodename: dv-cinder-855359-3-iscsi Sep 18 06:23:31.832589 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: machine: x86_64 Sep 18 06:23:31.832702 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: clock source: unix Sep 18 06:23:31.832811 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: pcre jit disabled Sep 18 06:23:31.832920 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: detected number of CPU cores: 4 Sep 18 06:23:31.833029 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: current working directory: / Sep 18 06:23:31.833167 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: detected binary path: /usr/bin/uwsgi-core Sep 18 06:23:31.833167 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: your processes number limit is 47680 Sep 18 06:23:31.833167 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: your memory page size is 4096 bytes Sep 18 06:23:31.833167 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: detected max file descriptor number: 2048 Sep 18 06:23:31.833167 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: lock engine: pthread robust mutexes Sep 18 06:23:31.833527 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: thunder lock: enabled Sep 18 06:23:31.833527 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 18 06:23:31.833671 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 18 06:23:31.873289 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: Python main interpreter initialized at 0x556e7bcdb9c0 Sep 18 06:23:31.873542 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: python threads support enabled Sep 18 06:23:31.873652 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: your server socket listen backlog is limited to 100 connections Sep 18 06:23:31.873758 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: your mercy for graceful operations on workers is 90 seconds Sep 18 06:23:31.874191 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: mapped 403077 bytes (393 KB) for 2 cores Sep 18 06:23:31.874416 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: *** Operational MODE: preforking *** Sep 18 06:23:31.874870 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: *** uWSGI is running in multiple interpreter mode *** Sep 18 06:23:31.875002 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: spawned uWSGI master process (pid: 120365) Sep 18 06:23:31.875345 dv-cinder-855359-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 18 06:23:31.877450 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: spawned uWSGI worker 1 (pid: 120380, cores: 1) Sep 18 06:23:31.877450 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: spawned uWSGI worker 2 (pid: 120381, cores: 1) Sep 18 06:23:31.877450 dv-cinder-855359-3-iscsi devstack@n-api.service[120365]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 18 06:23:32.540806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 18 06:23:32.541410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 18 06:23:33.698139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:23:33.698139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: from cryptography.utils import int_from_bytes Sep 18 06:23:33.699411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:23:33.699411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: from cryptography.utils import int_from_bytes Sep 18 06:23:33.704721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:23:33.704721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: from cryptography.utils import int_from_bytes Sep 18 06:23:33.705316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:23:33.705316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: from cryptography.utils import int_from_bytes Sep 18 06:23:36.064668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 06:23:36.065339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 06:23:36.065636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 06:23:36.065863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 06:23:36.066130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 06:23:36.066534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.066902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.067237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.067566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.067911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.068232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.068567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.068898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.069236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.069506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.069834 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.070170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-855359-3-iscsi {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.070504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.070853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.071183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.071514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.071853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.072170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.072494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.073043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.074837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.074982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.077840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.077840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.077840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.077840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.077840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.077840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.077840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.077840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.079288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-855359-3-iscsi {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.079288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.079288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.079525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.080024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.081523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.081523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.081523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.081523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.081523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.082291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.082291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.082635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.082820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.083653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.083653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.084143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.084143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.084425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.084661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.084927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.085247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.085580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.085907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.086238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.086499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.086789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.087118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.087449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.087775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.088082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.088398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.088712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.089029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.089344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.089660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.090001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.090289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.090642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.090987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.091304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.091634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.092050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.20 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.092309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.20 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.092635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.092977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.093302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.093619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.093948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.094336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.094675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.094992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.095407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.095610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.096061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.096269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.096529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.096845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.097155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.097483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.097810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.098122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.098439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.098779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.099096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.099410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.099725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.100038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.100352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.100666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.100984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.101344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.101609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.101920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.102246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.102605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.102910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.103225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.103538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.103850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.104178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.104494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.104808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.105138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.105535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.105757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.106098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.106389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.106726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.107051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.107356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.107672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.108045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.108316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 06:23:36.108836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.108993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 06:23:36.109147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.109147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.109426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 06:23:36.109568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.109764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 06:23:36.109910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.110114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 06:23:36.110275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.110426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.110686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.110836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.110980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.111271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.111421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.111574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.111717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.111903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.112060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.112213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.112373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.112524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.112673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.112816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.112971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.113114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.113263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.113414 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.113564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.113715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.113866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.114008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-855359-3-iscsi {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.114162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.114319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.114462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.114638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.114795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.114940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.115081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.115231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.115384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.115557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.115863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.116020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.116336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.116496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.116639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.116639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.117040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.117313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.117469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.117623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.117782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.117925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.118075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.118223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.118395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.118676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.118844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.118999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.119142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.119283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.119425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.119567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.119786 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.119930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.120117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.120272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.120425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.120593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.120748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-855359-3-iscsi {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.120891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.121041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.121200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.121515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.121673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.121817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.122007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.122163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.122306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.122462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.122640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.122801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.122951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.123125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.123283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.123444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.123600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.123750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.123898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.124049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.124200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.124352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.124505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.124667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.124810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.124951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.125107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.125799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.126131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.126425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.126632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.126797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.127040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.127197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.127435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.127593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.127748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.128088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.128333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.128479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.128647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.128895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.129050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.129209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.129353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.129504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.129647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.129796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.129957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.129957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.130248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.130391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.130540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.130719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.130719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.131006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.131151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.131151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.131445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.131775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.131775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.132139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.132296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.132296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.132587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.132737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.132737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.133015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.133157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.133306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.133455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.133455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.133742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.133894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.133894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.134171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.134321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.134749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.134899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.135041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.135184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.135333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.135483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.135635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.135777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.136153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.136327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.136647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.136647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.136935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.137190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.20 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.137190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.20 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.137590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.137751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.137895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.138055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.138210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.138355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.138504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.138754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.138930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.139218 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.139368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.139518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.139660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.139809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.139966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.140116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.140266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.140411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.140561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.140721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.140882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.141682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.142036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.142257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.142257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.142541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.142711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.142711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.142995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.143147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.143147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.143423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.143577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.143577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.143865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.144015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.144015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.144290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.144440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.144593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.144746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.144896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.145047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.145190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.145340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.145502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.145657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.145799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.145942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.145942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.146410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.146767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.146767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.147051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.147221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.147455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.147629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.147629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.148307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.148461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.148604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.148764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.148916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.149067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.149229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.149566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.149566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.149861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.150004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.150154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.150309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.150465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.150627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.150777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.150921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.151074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.151226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.151226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.151507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.151649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.151802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.151964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.152339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.152503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.152503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.152806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.153238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.153238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.153525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.153679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.153829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.153981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.154123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.154265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.154419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.154563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.154778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.154923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.155066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.155216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.155365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.155507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.155655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.155797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.155946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.156106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.156249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.156584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.156754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.157053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.157212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.157365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.157597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.157755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.157986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.158142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:23:36.158285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.158441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.158671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.158954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.159120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.159273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.159427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.159686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.159836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.159986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.160136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.160280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.160431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.160575 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.160725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.160867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.161008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.161158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.161327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.161472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.161613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.161762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.161907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.162056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.162230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.162552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.162744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.163014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.163165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.163328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.163473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.163626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.163776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.163919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.164061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.164212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.164355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.164506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.164649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.164791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.164939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.165945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.166177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.166366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.166484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.166743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.166833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.167125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.167220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.167357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.167449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.167678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.167779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.168396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.168496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.168496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.168651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.168724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.168938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.169063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.169246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.169397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.169674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.169767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.169948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.170053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.170366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.170473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.170559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.170780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.171134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.171259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.171339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.171421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.171525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.171892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.172185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.172332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.172714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.172827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.172904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.172974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.173185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.173279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.173514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.173658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.173901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.173992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.174285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.174389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.174602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.174789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.174960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.175154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.175350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.175453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.175597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.175924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.176069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.176262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.176372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.176824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.176943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.177040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.177283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.177380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.177473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.177565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.177759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.177884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.178040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.178217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.178372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.178515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.178747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.178842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.179074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.179167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.179404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.179506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.179727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.179826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.180084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.180187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.180479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.180678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.180768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.180851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.181001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.181096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.181307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.181406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.181619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.181717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.181936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.182035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.182249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.182349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.182672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.183902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.184136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.184227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.184459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.184548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.184801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.185035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.185120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.185384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.185467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.185710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.185792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.186037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.186132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.186350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.186430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.187020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.187342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.187514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.187667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.188537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.188693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.188855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.189000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.189142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.189284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.189435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.189594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.189747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.190023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.190199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.190429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.190610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.190779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.190991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.191140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.191283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.191425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.191567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.191725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.191868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.192019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.192172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.192314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.192579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.192768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.193027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.193182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.193326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.193467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.193707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.193873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.194018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.194169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.194332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.194482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.194661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.194818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.194963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.195106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.195263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.195407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.195559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.195709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.195861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.196004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.196146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.196310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.196467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.196611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.196761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.196911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.197248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.197523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.197688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.198074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.198244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.198417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.198738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.198896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.199053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.199196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.199338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.199480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.199629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.199800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.200140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.200340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.200597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.200753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.200904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.201062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.201207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.201370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.201514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.201655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.201815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.201977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.202119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.202270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.202413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.202554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.202741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.202894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.203143 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.203328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.204188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.204340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.204494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.204637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.204788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.204932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.205083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.205235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.205393 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.205537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.205689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.205832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.205999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.206151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.206354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.206511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.206732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.206885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.207029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.207360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.207360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.207790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.207942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.208093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.208237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.208402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.208564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.208853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.209010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.209161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.209305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.209456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.209600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.209742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.210014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.210174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.210318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.210471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.210647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.211005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.211158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.211312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.211466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.211612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.211755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.211898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.212047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.212190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.212342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.212493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.212651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.212812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.212964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.213233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.213398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.213551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.213703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.213848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.213997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.214147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.214406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.214562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.214775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.214940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.215083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.215303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.215474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.215753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.215903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.216054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.216198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.216350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.216510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.216774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.216923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.217073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.217365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.217527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.217776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.217925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.218068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.218219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.218362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.218514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.218680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.219000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.219173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.219318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.219469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.219626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.219776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.219920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.220063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.220213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.220365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.220508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.220650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.220800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.221049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.221205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.221370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.221529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.221694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.221844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.222002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.222154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.222316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.222466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.222645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.222801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.223105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.223378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.223613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.223774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.224643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.224814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.224968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.225117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.225277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.225436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.225596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.225755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.225907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.226101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.226277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.226440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.226621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.226812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.227054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.227054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.227527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.227703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.227946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.228111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.228270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.228428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.228578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.228737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.229206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.229385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.229549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.229709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.229867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.230018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.230175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.230333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.230494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.230666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.230820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.230970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.231241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.231417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.231579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.231738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.231898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.232057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.232215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.232373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.232578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.232735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.234199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.234371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.234524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.234831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.235018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.235178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.235377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.235535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.235685 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.235850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.236001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.236151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.236301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.236301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.236598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.236763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.236915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.237555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.237790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.237972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.238625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.238811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.238974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.239315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.239431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.239514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.239594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.239684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.239769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.239853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.239935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240370 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.240996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.241942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.242955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.243969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244685 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.244956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245211 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.245992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.246925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.247923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.248941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.249880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.250047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.250131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.250755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.250848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.250934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.251997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.252946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.253955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.254040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.254188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.254293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.254496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.254612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.254828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.254931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.255155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.255261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.255484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.255638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.255800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.255903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.256114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.256213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.256427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.256527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.256740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.256862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.257046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.257144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.257360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.257451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.257669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.257826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.257964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.258066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.258463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.258559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.258723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.258928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.259260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.259747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.260030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.260120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.260200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.260324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.260441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.260651 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.260755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.260977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.261107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.261292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.261425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.261611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.261764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.261932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.262083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.263585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.263727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.263815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.263896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.263989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.264923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.265013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.265098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.265188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.265273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.265365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.265539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.265632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.265881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.266569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.266882 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.267209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.267532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.267865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.268085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.268198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.268448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.268550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.268789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.268894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.269126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.269221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.269464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.269569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.269802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.269896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.270121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.270213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.270433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.270533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.270780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.270875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.271115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.271233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.271428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.271530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.271743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.271836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.272055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.272164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.272388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.272501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.272698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.272791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.273024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.273128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.273381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.273490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.273709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.273805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.274038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.274135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.274407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.274503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.274759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.274855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.275080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.275176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.275404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.275499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.275734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.275828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.276060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.276155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.276373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.276474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.276692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.276795 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.277009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.277122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.277334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.277439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.277654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.277759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.277975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.278075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.278296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.278397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.278718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.278843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.278941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.279037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.279276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.279380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.279598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.279700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.279927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.280021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.280250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.280343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.280579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.280681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.280975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.281117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.281310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.281441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.281667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.281809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.281981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.282144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.282294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.282476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.282704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.282829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.283022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.283156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.283336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.283489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.283664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.283804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.284007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.284144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.284327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.284463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.284645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.284780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.284961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.285093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.285279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.285422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.285589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.285741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.285918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.286054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.286228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.286381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.286559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.286749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.286906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.287239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.287343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.287567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.287673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.287880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.287981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.288208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.288314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.288526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.288621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.288838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.288950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.289150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.289255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.289462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.289602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.289813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.289908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.290127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.290236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.290449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.290549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.290797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.290892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.291115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.291219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.291430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.291526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.291751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.291845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.292070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.292165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.292385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.292516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.292705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.292809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.293018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.293113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.293345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.293447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.293657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.293752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.293967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.294061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.294277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.294371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.294635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.294750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.294918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.295020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.295227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.295319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.295540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.295643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.295848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.295940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.296155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.296248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.296466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.296573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.296779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.296871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.297088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.297200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.297439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.297639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.297779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.297868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.298036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.298145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.298350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.298464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.298686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.298795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.299030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.299135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.299337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.299440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.299658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.299761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.299969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.300063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.300295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.300397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.300616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.300710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.300938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.301032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.301258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.301353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.301577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.301681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.301896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.302121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.302224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.302461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.302555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.302819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.302932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.303158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.303252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.303477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.303570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.303794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.303888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.304111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.304350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.304445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.304687 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.304789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.305007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.305111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.305329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.305431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.305646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.305750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.305961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.306054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.306277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.306370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.306625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.306723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.306923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.307019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.307239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.307339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.307551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.307641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.307866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.307960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.308184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.308285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.308501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.308612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.308820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.308924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.309142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.309246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.309446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.309551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.309762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.309856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.310081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.310183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.310418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.310520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.310773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.310870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.311085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.311190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.311401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.311493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.311718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.311820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.312032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.312132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.312343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.312437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.312658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.313290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.313519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.313661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.313840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.313962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.314155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.314249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.314473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.314814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.314908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.315136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.315228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.315465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.315559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.315809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.315910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.316133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.316233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.316453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.316559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.316772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.316864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.317093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.317184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.317421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.317618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.317743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.317940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.318097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.318257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.318402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.318609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.318762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.318945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.319170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.319277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.319411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.319577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.319723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.319908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.320033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.320218 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.320352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.320528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.320679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.320843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.321003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.321155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.321316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.321465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.321627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.321778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.321945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.322087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.322260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.322396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.322620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.322729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.322909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.323042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.323223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.323542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.323633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.323937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.324087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.324194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.324283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.324461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.324602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.324774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.324870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.325085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.325180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.325395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.325498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.325710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.325805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.326021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.326124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.326333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.326428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.326664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.326776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.326978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.327073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.327288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.327381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.327594 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.327702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.327909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.328004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.328222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.328326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.329119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.329259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.329407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.329529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.329718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.329869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.330052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.330241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.330366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.330483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.330709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.330817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.331022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.331126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.331337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.331440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.331646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.331740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.331954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.332051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.332260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.332362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.332565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.332668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.332879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.332974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.333188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.333294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.333501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.333635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.333835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.333937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.334170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.334284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.334480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.334610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.334852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.334951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.335163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.335256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.335492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.335605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.335808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.335904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.336117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.336218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.336439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.336542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.336754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.336855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.337070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.337163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.337385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.337487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.337700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.337793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.338019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.338112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.338343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.338454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.338684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.338910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.339017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.339260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.339424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.339573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.339668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.339918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.340020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.340258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.340361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.340571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.340673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.340887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.340980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.341200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.341295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.341531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.341627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.341849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.341980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.342169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.342272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.342485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.342646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.342827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.342921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.343143 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.343238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.343455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.343555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.343707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 06:23:36.343818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.344110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.344430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.344744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.345059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.345375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.345699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.346015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.346335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.346710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.347040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.347363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.347684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.348008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.348324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.348640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.348956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.349302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.349586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.349921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.350240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.350550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.350994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.351312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.351678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.351960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.352276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.352602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.352928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.353243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.353569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.353885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.354219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.354566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.354920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.355249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.355566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.355876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.356185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.356500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.356823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.357137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.357446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.357754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.358067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.358376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.358711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.359022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.359329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.359711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.359969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.360276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.360582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.360887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.361192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.361499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.361810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.362117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.362424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.362753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.363060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.363366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.363702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.364040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.364360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.364691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.365007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.365334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.365652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.365979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.366305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.366646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.366961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.367299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.367614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.367929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.368248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.368565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.368881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.369255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.369527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.369856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.370163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.370492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.370840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.371165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.371485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.371800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.372135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.372453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.372767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.373081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.373396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.373710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.374031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.374344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.374690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.375016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.375852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.376193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.376523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.376874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.377161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.377491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.377840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.378156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.378503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.378861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.379227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.379498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.379827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:23:36.380048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 06:23:36.443627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO dbcounter [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Registered counter for database nova_api Sep 18 06:23:36.468020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_db.sqlalchemy.engines [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120380) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:23:36.474753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writer thread running {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:23:36.495777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO dbcounter [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Registered counter for database nova_api Sep 18 06:23:36.516371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_db.sqlalchemy.engines [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120381) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:23:36.522910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writer thread running {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:23:36.531269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.context [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),63c4bc6f-f107-41b9-b2c1-7478fed24e9a(cell1) {{(pid=120380) load_cells /opt/stack/nova/nova/context.py:464}} Sep 18 06:23:36.535341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:23:36.535823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:23:36.537390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:23:36.538178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:23:36.538628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:23:36.540625 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:23:36.542652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO dbcounter [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Registered counter for database nova_cell0 Sep 18 06:23:36.546672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO dbcounter [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Registered counter for database nova_cell1 Sep 18 06:23:36.554489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_db.sqlalchemy.engines [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120380) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:23:36.556307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_db.sqlalchemy.engines [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120380) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:23:36.562553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writer thread running {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:23:36.564587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writer thread running {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:23:36.580500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.context [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),63c4bc6f-f107-41b9-b2c1-7478fed24e9a(cell1) {{(pid=120381) load_cells /opt/stack/nova/nova/context.py:464}} Sep 18 06:23:36.584418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:23:36.584902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:23:36.586473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:23:36.587247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:23:36.587703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:23:36.589678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:23:36.591711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO dbcounter [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Registered counter for database nova_cell0 Sep 18 06:23:36.596108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO dbcounter [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Registered counter for database nova_cell1 Sep 18 06:23:36.603515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_db.sqlalchemy.engines [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120381) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:23:36.604742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_db.sqlalchemy.engines [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120381) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:23:36.611833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writer thread running {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:23:36.614755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writer thread running {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:23:36.711269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO dbcounter [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Registered counter for database nova_cell0 Sep 18 06:23:36.719184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_db.sqlalchemy.engines [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120380) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:23:36.723795 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writer thread running {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:23:36.759998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO dbcounter [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Registered counter for database nova_cell0 Sep 18 06:23:36.769727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_db.sqlalchemy.engines [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120381) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:23:36.773795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writer thread running {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:23:37.041939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: WARNING keystonemiddleware.auth_token [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 06:23:37.049761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: WARNING keystonemiddleware.auth_token [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 06:23:37.094793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 06:23:37.102945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 06:23:37.391752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: WARNING keystonemiddleware.auth_token [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 06:23:37.398635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: WARNING keystonemiddleware.auth_token [None req-6ffe7c44-ca59-4c29-9faf-30dace06f25b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 06:23:37.404346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556e7bcdb9c0 pid: 120380 (default app) Sep 18 06:23:37.459031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 06:23:37.465082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-6fc9b0e3-a839-49b5-a630-9a2ce5ffe08e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 06:23:37.470945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556e7bcdb9c0 pid: 120381 (default app) Sep 18 06:24:13.220165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=3 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:13.222173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=3 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:13.223280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=3 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:13.224440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=3 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:14.502079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fcce3880-ad11-4cf7-9e08-878adfde79dd tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Action: 'create', calling method: .version_select at 0x7fe6713fc4c0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1629537586", "metadata": {"Type": "work"}}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fcce3880-ad11-4cf7-9e08-878adfde79dd tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Request method failure captured: Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 122 Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmN6E8brIrlV-4vcNcVScF5l4kCeDSZqps6yFk4vAPoLqCGiDzO8IHmK8yrBRdJO1fi6U4JfSAS5Cr0_Ovus0xZPC753jx1Mvhe8OjsNaZQqhBIHFL18JwEUiHywTST2rZz65EGefjQDfeXcmVHf3gHljvp1MHz19nVhQ0sT8nHwC4 Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:24:14.513819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 85bf4e10a7674509961e17a3ed345f9b Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1629537586", "metadata": {"Type": "work"}}} Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: Sep 18 06:24:14.515841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1629537586', 'metadata': {'Type': 'work'}}}} Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise error Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi '#$%' Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:24:14.516811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 18 06:24:14.518605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 18 06:24:14.520738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 18 06:24:14.520738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.520738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fcce3880-ad11-4cf7-9e08-878adfde79dd tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:24:14.521630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fcce3880-ad11-4cf7-9e08-878adfde79dd tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 1.318130 Sep 18 06:24:14.522300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1/1] 10.222.0.20 () {60 vars in 1156 bytes} [Sun Sep 18 06:24:13 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 1326 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:24:14.912508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2f1f8874-b335-453b-876e-6b93e7b580fd tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Action: 'create', calling method: .version_select at 0x7fe671447dc0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1741439766", "metadata": {"Type": "work"}}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2f1f8874-b335-453b-876e-6b93e7b580fd tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Request method failure captured: Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 120 Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmN6E8brIrlV-4vcNcVScF5l4kCeDSZqps6yFk4vAPoLqCGiDzO8IHmK8yrBRdJO1fi6U4JfSAS5Cr0_Ovus0xZPC753jx1Mvhe8OjsNaZQqhBIHFL18JwEUiHywTST2rZz65EGefjQDfeXcmVHf3gHljvp1MHz19nVhQ0sT8nHwC4 Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:24:14.919125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 85bf4e10a7674509961e17a3ed345f9b Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1741439766", "metadata": {"Type": "work"}}} Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:24:14.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1741439766', 'metadata': {'Type': 'work'}}}} Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise error Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi '0' Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:24:14.921622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 18 06:24:14.923242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 18 06:24:14.924591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 18 06:24:14.924591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.930757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2f1f8874-b335-453b-876e-6b93e7b580fd tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:24:14.931680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2f1f8874-b335-453b-876e-6b93e7b580fd tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.403098 Sep 18 06:24:14.932566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1/2] 10.222.0.20 () {60 vars in 1156 bytes} [Sun Sep 18 06:24:14 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 405 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:24:14.942805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4dc689eb-6d89-49f6-8528-950d73d7eed2 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Action: 'create', calling method: .version_select at 0x7fe6713fcdc0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2108572169", "metadata": {"Type": "work"}}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4dc689eb-6d89-49f6-8528-950d73d7eed2 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Request method failure captured: Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 119 Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmN6E8brIrlV-4vcNcVScF5l4kCeDSZqps6yFk4vAPoLqCGiDzO8IHmK8yrBRdJO1fi6U4JfSAS5Cr0_Ovus0xZPC753jx1Mvhe8OjsNaZQqhBIHFL18JwEUiHywTST2rZz65EGefjQDfeXcmVHf3gHljvp1MHz19nVhQ0sT8nHwC4 Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:24:14.947483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 85bf4e10a7674509961e17a3ed345f9b Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2108572169", "metadata": {"Type": "work"}}} Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: Sep 18 06:24:14.948916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-2108572169', 'metadata': {'Type': 'work'}}}} Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise error Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi '' Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:24:14.949673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 18 06:24:14.951039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 18 06:24:14.952762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 18 06:24:14.952762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:14.955679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4dc689eb-6d89-49f6-8528-950d73d7eed2 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:24:14.957920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4dc689eb-6d89-49f6-8528-950d73d7eed2 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.018134 Sep 18 06:24:14.959861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 2/3] 10.222.0.20 () {60 vars in 1156 bytes} [Sun Sep 18 06:24:14 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 23 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 18 06:24:14.974084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=3 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:14.975202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=3 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:14.976006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=3 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:14.977160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=3 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:14.985243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0d631f00-49aa-42a9-b8f7-af36b7770c1b tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Calling method '.version_select at 0x7fe6713df5e0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:15.671556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-0d631f00-49aa-42a9-b8f7-af36b7770c1b tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] HTTP exception thrown: Volume tempest-invalid-32706683 could not be found. Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0d631f00-49aa-42a9-b8f7-af36b7770c1b tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Request method failure captured: Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-32706683 HTTP/1.1 Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmN6E8brIrlV-4vcNcVScF5l4kCeDSZqps6yFk4vAPoLqCGiDzO8IHmK8yrBRdJO1fi6U4JfSAS5Cr0_Ovus0xZPC753jx1Mvhe8OjsNaZQqhBIHFL18JwEUiHywTST2rZz65EGefjQDfeXcmVHf3gHljvp1MHz19nVhQ0sT8nHwC4 Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:24:15.675671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 85bf4e10a7674509961e17a3ed345f9b Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Volume tempest-invalid-32706683 could not be found. Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'tempest-invalid-32706683'} Sep 18 06:24:15.677065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-32706683 could not be found. (HTTP 404) (Request-ID: req-a1ef21e6-ae8c-467f-a486-4256b26947db) Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:15.678008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 06:24:15.679321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-32706683 could not be found. Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:24:15.681768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-32706683 could not be found. Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-32706683 could not be found. Sep 18 06:24:15.683147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:15.684939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0d631f00-49aa-42a9-b8f7-af36b7770c1b tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Returning 404 to user: Volume tempest-invalid-32706683 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:24:15.687873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0d631f00-49aa-42a9-b8f7-af36b7770c1b tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/tempest-invalid-32706683" status: 404 len: 97 microversion: 2.1 time: 0.721039 Sep 18 06:24:15.688949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1212 bytes} [Sun Sep 18 06:24:14 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-32706683 => generated 97 bytes in 725 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 18 06:24:15.699400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-494abc08-65bf-465b-8c7d-d566f33a0cb5 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003965 Sep 18 06:24:15.701024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 3/5] 10.222.0.20 () {58 vars in 1140 bytes} [Sun Sep 18 06:24:15 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 18 06:24:15.713318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-19b59c5c-f896-456e-88fc-b1659d567c20 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.007026 Sep 18 06:24:15.715385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 3/6] 10.222.0.20 () {58 vars in 1137 bytes} [Sun Sep 18 06:24:15 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 10 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 18 06:24:15.724768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-358ad6c0-0c82-4204-abac-86dfd6a2e353 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Calling method '.version_select at 0x7fe6712bc430>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:15.851934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-88649e39-420e-4efa-84f3-3fdd116459c1 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe671215040>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:15.975087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-88649e39-420e-4efa-84f3-3fdd116459c1 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/db194064-6c95-4ba4-b7d6-286dc2c383de" status: 200 len: 351 microversion: 2.1 time: 0.127048 Sep 18 06:24:15.977052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 06:24:15 2022] GET /compute/v2.1/os-volumes/db194064-6c95-4ba4-b7d6-286dc2c383de => generated 351 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:16.086386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-358ad6c0-0c82-4204-abac-86dfd6a2e353 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] HTTP exception thrown: Volume 69a43c60-e14b-49cd-a67d-e6925d1c4c53 could not be found. Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-358ad6c0-0c82-4204-abac-86dfd6a2e353 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Request method failure captured: Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/os-volumes/69a43c60-e14b-49cd-a67d-e6925d1c4c53 HTTP/1.1 Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmN6E8brIrlV-4vcNcVScF5l4kCeDSZqps6yFk4vAPoLqCGiDzO8IHmK8yrBRdJO1fi6U4JfSAS5Cr0_Ovus0xZPC753jx1Mvhe8OjsNaZQqhBIHFL18JwEUiHywTST2rZz65EGefjQDfeXcmVHf3gHljvp1MHz19nVhQ0sT8nHwC4 Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:24:16.095136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 85bf4e10a7674509961e17a3ed345f9b Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Volume 69a43c60-e14b-49cd-a67d-e6925d1c4c53 could not be found. Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '69a43c60-e14b-49cd-a67d-e6925d1c4c53'} Sep 18 06:24:16.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 69a43c60-e14b-49cd-a67d-e6925d1c4c53 could not be found. (HTTP 404) (Request-ID: req-42f137dd-5dc7-4dd8-9bd0-1d161417ea14) Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:16.097517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 06:24:16.099762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 69a43c60-e14b-49cd-a67d-e6925d1c4c53 could not be found. Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 18 06:24:16.102498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 69a43c60-e14b-49cd-a67d-e6925d1c4c53 could not be found. Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 69a43c60-e14b-49cd-a67d-e6925d1c4c53 could not be found. Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-358ad6c0-0c82-4204-abac-86dfd6a2e353 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Returning 404 to user: Volume 69a43c60-e14b-49cd-a67d-e6925d1c4c53 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-358ad6c0-0c82-4204-abac-86dfd6a2e353 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/69a43c60-e14b-49cd-a67d-e6925d1c4c53" status: 404 len: 109 microversion: 2.1 time: 0.381228 Sep 18 06:24:16.104995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1248 bytes} [Sun Sep 18 06:24:15 2022] DELETE /compute/v2.1/os-volumes/69a43c60-e14b-49cd-a67d-e6925d1c4c53 => generated 109 bytes in 383 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:24:16.113536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bb52c6c7-d012-4d3d-a61c-3277865cddb3 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Calling method '.version_select at 0x7fe671215790>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:16.201911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-bb52c6c7-d012-4d3d-a61c-3277865cddb3 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] HTTP exception thrown: Volume 18edb4c5-823c-44b9-a187-943bed9ea7b9 could not be found. Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bb52c6c7-d012-4d3d-a61c-3277865cddb3 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Request method failure captured: Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/os-volumes/18edb4c5-823c-44b9-a187-943bed9ea7b9 HTTP/1.1 Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmN6E8brIrlV-4vcNcVScF5l4kCeDSZqps6yFk4vAPoLqCGiDzO8IHmK8yrBRdJO1fi6U4JfSAS5Cr0_Ovus0xZPC753jx1Mvhe8OjsNaZQqhBIHFL18JwEUiHywTST2rZz65EGefjQDfeXcmVHf3gHljvp1MHz19nVhQ0sT8nHwC4 Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:24:16.204352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/dade588a399b4216aa97e0f86e509ba3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: dade588a399b4216aa97e0f86e509ba3 Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesNegativeTest-71131319 Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 85bf4e10a7674509961e17a3ed345f9b Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesNegativeTest-71131319-project Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Volume 18edb4c5-823c-44b9-a187-943bed9ea7b9 could not be found. Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '18edb4c5-823c-44b9-a187-943bed9ea7b9'} Sep 18 06:24:16.205662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 18edb4c5-823c-44b9-a187-943bed9ea7b9 could not be found. (HTTP 404) (Request-ID: req-25dca8dd-5def-4e1a-9e78-b89c675e2f3a) Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:16.206444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 06:24:16.207724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 18edb4c5-823c-44b9-a187-943bed9ea7b9 could not be found. Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 18 06:24:16.210163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 18edb4c5-823c-44b9-a187-943bed9ea7b9 could not be found. Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 18edb4c5-823c-44b9-a187-943bed9ea7b9 could not be found. Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bb52c6c7-d012-4d3d-a61c-3277865cddb3 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] Returning 404 to user: Volume 18edb4c5-823c-44b9-a187-943bed9ea7b9 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:24:16.213266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bb52c6c7-d012-4d3d-a61c-3277865cddb3 tempest-VolumesNegativeTest-71131319 tempest-VolumesNegativeTest-71131319-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/18edb4c5-823c-44b9-a187-943bed9ea7b9" status: 404 len: 109 microversion: 2.1 time: 0.098152 Sep 18 06:24:16.219884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 5/9] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 06:24:16 2022] GET /compute/v2.1/os-volumes/18edb4c5-823c-44b9-a187-943bed9ea7b9 => generated 109 bytes in 112 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:24:18.027657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2a67d3b1-19d6-4877-9cf5-e6d26895c9a0 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe671241f70>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:18.115337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2a67d3b1-19d6-4877-9cf5-e6d26895c9a0 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/c1e8567d-56f3-44c8-96be-f88b8d5c5341" status: 200 len: 351 microversion: 2.1 time: 0.091813 Sep 18 06:24:18.116095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 5/10] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 06:24:18 2022] GET /compute/v2.1/os-volumes/c1e8567d-56f3-44c8-96be-f88b8d5c5341 => generated 351 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:24:19.764552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-63ebd385-d1c4-46b1-ae8c-4e27133e9d2d tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe671215670>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:19.900034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-63ebd385-d1c4-46b1-ae8c-4e27133e9d2d tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/37a85129-0799-4628-bef1-201d74ef23dd" status: 200 len: 351 microversion: 2.1 time: 0.142706 Sep 18 06:24:19.900942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 6/11] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 06:24:19 2022] GET /compute/v2.1/os-volumes/37a85129-0799-4628-bef1-201d74ef23dd => generated 351 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:19.915191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bdb5a706-e178-4c05-9e24-c1741d57aa82 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe6711f05e0>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:20.112765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bdb5a706-e178-4c05-9e24-c1741d57aa82 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.201958 Sep 18 06:24:20.113595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1134 bytes} [Sun Sep 18 06:24:19 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:20.128977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cf03db85-095b-411f-8174-a340b4e9f5dc tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe671215c10>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:20.334949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cf03db85-095b-411f-8174-a340b4e9f5dc tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.209695 Sep 18 06:24:20.335517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1149 bytes} [Sun Sep 18 06:24:20 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:20.351712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-932c27ea-3784-46db-afd2-84857874bf05 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe6713e0b80>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:20.507697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-932c27ea-3784-46db-afd2-84857874bf05 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.160909 Sep 18 06:24:20.509348 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 7/14] 10.222.0.20 () {58 vars in 1134 bytes} [Sun Sep 18 06:24:20 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:20.523433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-db28bcaa-ac52-4a41-9b0d-0ba8170cd13b tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe671215160>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:20.656363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-db28bcaa-ac52-4a41-9b0d-0ba8170cd13b tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.137299 Sep 18 06:24:20.661449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 8/15] 10.222.0.20 () {58 vars in 1167 bytes} [Sun Sep 18 06:24:20 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:20.675277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3f6fcd96-c6a3-40e9-9aeb-f96584e0a3cc tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe67129e040>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:20.889223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3f6fcd96-c6a3-40e9-9aeb-f96584e0a3cc tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.221772 Sep 18 06:24:20.889876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1170 bytes} [Sun Sep 18 06:24:20 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:20.907388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3cd34d6d-990f-45e6-a10a-ca2f2fd0ba11 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe6711a9940>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:21.153063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3cd34d6d-990f-45e6-a10a-ca2f2fd0ba11 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.252956 Sep 18 06:24:21.154047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 9/17] 10.222.0.20 () {58 vars in 1155 bytes} [Sun Sep 18 06:24:20 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:21.170486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d7565e33-1143-41be-aa87-6ca86f8c8ec4 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe6711ae700>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:21.341528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d7565e33-1143-41be-aa87-6ca86f8c8ec4 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.176683 Sep 18 06:24:21.342775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 10/18] 10.222.0.20 () {58 vars in 1188 bytes} [Sun Sep 18 06:24:21 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:21.355886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d6dcf96a-4dfb-44c5-92cb-c79e3751c861 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] Calling method '.version_select at 0x7fe6711e78b0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:21.486462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1382096120", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:21.503653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d6dcf96a-4dfb-44c5-92cb-c79e3751c861 tempest-VolumesTestJSON-808155217 tempest-VolumesTestJSON-808155217-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.151358 Sep 18 06:24:21.504660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 11/19] 10.222.0.20 () {58 vars in 1155 bytes} [Sun Sep 18 06:24:21 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:24:21.788067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] validate_networks() for [('0831a5b2-a7c5-4c5e-ba2b-086694d4e37b', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:24:23.572614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:24:23.573707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:24:23.574339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:24:23.574908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:24:23.575508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:24:23.612313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:23.613095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:23.613714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:23.616791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:23.617394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:23.617833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:23.682200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Getting quotas for project 91ef3478fd574180abe99393ba769124. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:24:23.711065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Getting quotas for user 7a6b8fbd31464e4d99449c74b4d1dda5 and project 91ef3478fd574180abe99393ba769124. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:24:23.744618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:24:23.756218 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:24:23.955025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0c0d00b6-3da0-44e1-b670-413b6f180d94 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.868723 Sep 18 06:24:23.972080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 9/20] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:24:21 2022] POST /compute/v2.1/servers => generated 383 bytes in 2886 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:24:23.983496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-092c6e43-25ce-498e-b10d-fe6c3c44549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:23.984535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-092c6e43-25ce-498e-b10d-fe6c3c44549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:24.277103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-092c6e43-25ce-498e-b10d-fe6c3c44549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:24.305496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-092c6e43-25ce-498e-b10d-fe6c3c44549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1222 microversion: 2.1 time: 0.325592 Sep 18 06:24:24.313958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 12/21] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:23 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1222 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:24.597947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1678363851", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:24.720475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] validate_networks() for [('0831a5b2-a7c5-4c5e-ba2b-086694d4e37b', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:24:25.897018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:24:25.897695 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:24:25.898219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:24:25.898844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:24:25.899807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:24:25.920295 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:25.923134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:25.925210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:25.936608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:25.937184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:25.937633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:25.991679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Getting quotas for project 91ef3478fd574180abe99393ba769124. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:24:26.015673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Getting quotas for user 7a6b8fbd31464e4d99449c74b4d1dda5 and project 91ef3478fd574180abe99393ba769124. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:24:26.064094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:24:26.068300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:24:26.193208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cda1880e-8dac-4d22-9145-d7c9c630691a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.599644 Sep 18 06:24:26.194457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 10/22] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:24:24 2022] POST /compute/v2.1/servers => generated 383 bytes in 1602 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:24:26.206783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9aec241d-66dd-4dbc-9c5b-e41ac08dff31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:26.209287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9aec241d-66dd-4dbc-9c5b-e41ac08dff31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:26.327461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9aec241d-66dd-4dbc-9c5b-e41ac08dff31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:26.339841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9aec241d-66dd-4dbc-9c5b-e41ac08dff31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1222 microversion: 2.1 time: 0.137827 Sep 18 06:24:26.345367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 13/23] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:26 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1222 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:24:27.366470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-59c07f8f-383d-4eae-85ec-00c62a2269b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:27.367996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-59c07f8f-383d-4eae-85ec-00c62a2269b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:27.537251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-59c07f8f-383d-4eae-85ec-00c62a2269b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:27.560024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-59c07f8f-383d-4eae-85ec-00c62a2269b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1222 microversion: 2.1 time: 0.198335 Sep 18 06:24:27.562230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 11/24] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:27 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1222 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:28.606342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:28.611803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:28.668422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:28.672412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:28.675015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:28.680280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:28.689205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:28.690754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:29.271888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:29.306301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-133b12e7-198c-45e5-89ac-667f5ce837ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1218 microversion: 2.1 time: 0.712700 Sep 18 06:24:29.323809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 14/25] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:28 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1218 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:30.341055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:30.343645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:30.379601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:30.380784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:30.382558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:30.383681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:30.384936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:30.386924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:31.589599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:31.635674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-78e4a9e3-4390-4810-b6ce-e09635cd4bed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1280 microversion: 2.1 time: 1.298741 Sep 18 06:24:31.637633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 12/26] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:30 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1280 bytes in 1305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:32.672174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:32.673615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:32.692016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:32.694613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:32.695378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:32.702709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:32.706147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:32.706925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:33.078505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:33.089297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5864ba58-feaa-415d-89dc-3ef15289431f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1280 microversion: 2.1 time: 0.431522 Sep 18 06:24:33.089848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 15/27] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:32 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1280 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:34.125781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:34.128505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:34.148014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:34.150276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:34.152102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:34.153088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:34.153998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:34.154644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:34.379910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:34.399366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6925a832-01a3-46dd-b313-2622143a181b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1280 microversion: 2.1 time: 0.276409 Sep 18 06:24:34.416233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 13/28] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:34 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1280 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:35.426522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:35.427874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:35.438827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:35.439785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:35.441883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:35.443751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:35.445652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:35.447523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:36.334082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:37.275421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:37.288387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9072b099-c206-4103-9468-f1c0bb6989ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 1.865880 Sep 18 06:24:37.304371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 16/29] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:35 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 1882 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:38.313658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:38.315303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:38.317685 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:38.319376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=14 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:38.337274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:38.342878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:38.345941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:38.347623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:38.358671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:38.359580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:38.738807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:38.747301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a275bb48-f498-4e1b-9bf3-535079feb7ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.439528 Sep 18 06:24:38.760199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 14/30] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:38 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:39.068641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-ce3cfc74-1e79-4735-91f2-454e52f3245e req-607c5bb7-8d31-4fca-9d99-9f93572aa760 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1dd95cf5-d6be-4de9-9941-8b1d35855bcb", "tag": "76758f98-a1a3-4d17-a73a-67726f56e2d5"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:39.082438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-ce3cfc74-1e79-4735-91f2-454e52f3245e req-607c5bb7-8d31-4fca-9d99-9f93572aa760 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:39.082909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-ce3cfc74-1e79-4735-91f2-454e52f3245e req-607c5bb7-8d31-4fca-9d99-9f93572aa760 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:39.083339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-ce3cfc74-1e79-4735-91f2-454e52f3245e req-607c5bb7-8d31-4fca-9d99-9f93572aa760 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:39.109311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-ce3cfc74-1e79-4735-91f2-454e52f3245e req-607c5bb7-8d31-4fca-9d99-9f93572aa760 service nova] Creating event network-changed:76758f98-a1a3-4d17-a73a-67726f56e2d5 for instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb on hv-cinder-85535 Sep 18 06:24:39.110070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-ce3cfc74-1e79-4735-91f2-454e52f3245e req-607c5bb7-8d31-4fca-9d99-9f93572aa760 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120381) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 18 06:24:39.110645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-ce3cfc74-1e79-4735-91f2-454e52f3245e req-607c5bb7-8d31-4fca-9d99-9f93572aa760 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120381) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 18 06:24:39.112180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:39.112828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:39.113412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:39.114447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:39.115078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:39.115664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:39.128163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.rpcapi [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 18 06:24:39.129496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] Releasing lock "compute-rpcapi-router" {{(pid=120381) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 18 06:24:39.164573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-14f37c7d-423f-46d3-88b9-300c5fffdf8e None None] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.288803 Sep 18 06:24:39.166070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 17/31] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:24:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:39.774704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:39.776656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:39.783190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:39.783807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:39.784509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:39.785241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:39.785749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:39.786202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:39.970940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:39.979160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ff70ce99-29a8-4b8f-9efb-358d489d5e91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.209064 Sep 18 06:24:39.979737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 15/32] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:39 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:41.002701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:41.004175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:41.018694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:41.019199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:41.022788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:41.024980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:41.025777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:41.027414 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:41.224589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1780856201", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "networks": [{"uuid": "05b82eaf-adbf-4a57-bb2f-7716d50c30e1"}]}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:41.264161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:41.272852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0c609ce1-f089-4a35-9369-fe5b7815e1e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.274261 Sep 18 06:24:41.273779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 16/33] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:40 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:41.428940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] validate_networks() for [('05b82eaf-adbf-4a57-bb2f-7716d50c30e1', None, None, None, None, None)] {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:24:42.299270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:42.301035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:42.324662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:42.329700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:42.331225 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:42.333959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:42.336518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:42.339206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:42.737195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:42.747851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-68873308-dbd4-4990-a090-0aabdd31887f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.453852 Sep 18 06:24:42.748454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 17/34] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:42 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:24:42.922809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-b90053c9-e612-4465-865c-f2a998700ebf req-ec03e1fc-e63e-4e2f-bd61-b0997f6376a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d8351557-8730-42b4-9788-684192b14e98", "tag": "b4df9936-75d5-4973-a602-6e13994ffd2a"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:42.951807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-b90053c9-e612-4465-865c-f2a998700ebf req-ec03e1fc-e63e-4e2f-bd61-b0997f6376a4 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:42.955658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-b90053c9-e612-4465-865c-f2a998700ebf req-ec03e1fc-e63e-4e2f-bd61-b0997f6376a4 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:42.957297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-b90053c9-e612-4465-865c-f2a998700ebf req-ec03e1fc-e63e-4e2f-bd61-b0997f6376a4 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:43.027022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-b90053c9-e612-4465-865c-f2a998700ebf req-ec03e1fc-e63e-4e2f-bd61-b0997f6376a4 service nova] Creating event network-changed:b4df9936-75d5-4973-a602-6e13994ffd2a for instance d8351557-8730-42b4-9788-684192b14e98 on hv-cinder-85535 Sep 18 06:24:43.027681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-b90053c9-e612-4465-865c-f2a998700ebf req-ec03e1fc-e63e-4e2f-bd61-b0997f6376a4 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120380) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 18 06:24:43.029811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-b90053c9-e612-4465-865c-f2a998700ebf req-ec03e1fc-e63e-4e2f-bd61-b0997f6376a4 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120380) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 18 06:24:43.030964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:43.031441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:43.031877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:43.032607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:43.033057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:43.038866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:43.056809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.compute.rpcapi [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 18 06:24:43.063327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] Releasing lock "compute-rpcapi-router" {{(pid=120380) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 18 06:24:43.094156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c8dbc563-18cb-44c3-a8ef-c6f1409bd381 None None] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.334967 Sep 18 06:24:43.099495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 18/35] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:24:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:43.211233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:24:43.212448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:24:43.212937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:24:43.213367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:24:43.213771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:24:43.228473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cd55ed3b-dd8d-4f96-bcb7-894ccc274cc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: .version_select at 0x7fe670808af0>, body: {"keypair": {"name": "tempest-keypair-30015041", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBF5J6VsQp/v5Nn9nLUVxKhNaWZ8QmqIuey9l6go6n9n9Le2mHOB8CDyRC/MHd8OrR07l1wjUNRQlW2B69Aq7DJzcwsPE8BxsLofB+ujrA9axNyYd01Kks3sFCbuhjhmXDg=="}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:43.258868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:43.258868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:43.258868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:43.258868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:43.259184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:43.259380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:43.265891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=46,nova_api:INSERT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:43.300200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-cd55ed3b-dd8d-4f96-bcb7-894ccc274cc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:24:43.311202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-cd55ed3b-dd8d-4f96-bcb7-894ccc274cc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for user d85160b01d824bc3a32b425fab98d2c7 and project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:24:43.321264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Getting quotas for project 22ba57c2ca3e4ff9a4b77d8cecbfa4e9. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:24:43.331043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Getting quotas for user 93d23721dfd34020b659707b65eb2c33 and project 22ba57c2ca3e4ff9a4b77d8cecbfa4e9. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:24:43.358674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:24:43.358674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:24:43.381720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cd55ed3b-dd8d-4f96-bcb7-894ccc274cc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.280877 Sep 18 06:24:43.382441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 19/36] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:24:43 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:43.461119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c98fc406-f1d1-4acc-995b-fcc1cab32ef2 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'create', calling method: .version_select at 0x7fe67085eb80>, body: {"keypair": {"name": "tempest-keypair-1945423021", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMjlJ2oPMyVyu3GWWhw98wX0ZfJOZq7W2a6E1BzUG2PAmN4KgZCZfmG0E6Ggk5ESQagsotyzms9tkAjACpYux2chUjidv6WO4vISyrkV8sbOMh7+tMoCuzojAo1hswDI7g=="}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:43.486196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8748826b-c61f-4252-9c49-0066776d638b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.500717 Sep 18 06:24:43.498120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 18/37] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:24:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 2514 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:24:43.504201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bd99215a-455d-4c25-86b5-5f0bef1d676d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:43.504893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bd99215a-455d-4c25-86b5-5f0bef1d676d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:43.510396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-c98fc406-f1d1-4acc-995b-fcc1cab32ef2 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Getting quotas for project 9cbc859294e9456eb2ca751279a17f52. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:24:43.536827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-c98fc406-f1d1-4acc-995b-fcc1cab32ef2 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Getting quotas for user 6c945a3e6a4946559aeadc63566d29f4 and project 9cbc859294e9456eb2ca751279a17f52. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:24:43.578884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c98fc406-f1d1-4acc-995b-fcc1cab32ef2 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.186270 Sep 18 06:24:43.579707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 20/38] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:24:43 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:43.659127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bd99215a-455d-4c25-86b5-5f0bef1d676d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:43.671304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bd99215a-455d-4c25-86b5-5f0bef1d676d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1215 microversion: 2.1 time: 0.171142 Sep 18 06:24:43.674423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 19/39] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:43 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1215 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:43.775233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:43.776006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:43.785041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:43.795919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:43.796440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:43.797174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:43.797652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:43.798089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:44.220557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:44.236692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-456e3ef2-4968-4744-a894-2d57ca2ebff4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.464020 Sep 18 06:24:44.238245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 21/40] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:43 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:44.699906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:44.701127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:44.708409 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:44.709129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:44.709737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:44.710596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:44.711191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:44.711746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:44.785702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:44.795000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eff19717-e40d-4e76-9df4-09cec3d158d4 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1219 microversion: 2.1 time: 0.098852 Sep 18 06:24:44.811535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 20/41] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:44 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1219 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:45.266398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:45.267114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:45.275766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:45.296344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:45.296868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:45.298602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:45.299304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:45.299839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:45.609098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:45.620187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bb091c2d-334e-4986-af22-6d4b0e377f9d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.358223 Sep 18 06:24:45.620787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:45 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:45.819845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:45.820626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:45.822667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=3 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:45.842073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:45.842952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:45.843583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:45.844484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:45.845095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:45.845687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:45.967772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:45.991453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c1ce870d-8e67-4bbb-bd67-f0439a4db72c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1219 microversion: 2.1 time: 0.175663 Sep 18 06:24:45.992036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 21/43] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:45 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1219 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:46.647929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:46.648693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:46.660773 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:46.663843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:46.664732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:46.667070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:46.667687 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:46.669544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:46.828483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-330441212", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d572beb8-e9a5-4e8d-a0f6-5ec49c88e496"}], "security_groups": [{"name": "tempest-securitygroup--808037242"}], "key_name": "tempest-keypair-30015041", "user_data": "}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:46.846864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:46.865716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cbe993a8-1901-4143-855a-2c3b40a7cf31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.221401 Sep 18 06:24:46.866568 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 23/44] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:46 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:46.911425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-377467751", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "65bb78e9-94da-42c7-9b2e-c8fc52cca325"}], "security_groups": [{"name": "tempest-securitygroup--411283652"}], "key_name": "tempest-keypair-1945423021", "user_data": "}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:47.013477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] validate_networks() for [('d572beb8-e9a5-4e8d-a0f6-5ec49c88e496', None, None, None, None, None)] {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:24:47.090032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] validate_networks() for [('65bb78e9-94da-42c7-9b2e-c8fc52cca325', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:24:47.895035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:24:47.898381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:24:47.899089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:24:47.899826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:24:47.900317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:24:47.906062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:24:47.907860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:24:47.908571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:24:47.909363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:24:47.912970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:24:47.914627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:47.915593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:47.916116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:47.916977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:47.917727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:47.918424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:47.924796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:47.925297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:47.925739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:47.926469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:47.927016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:47.927748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:47.945427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Getting quotas for project 9cbc859294e9456eb2ca751279a17f52. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:24:47.951465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Getting quotas for user 6c945a3e6a4946559aeadc63566d29f4 and project 9cbc859294e9456eb2ca751279a17f52. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:24:47.971693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:24:47.971913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:24:47.976284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:24:47.976772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for user d85160b01d824bc3a32b425fab98d2c7 and project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:24:47.997059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:24:47.999712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:24:48.045750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6f298e6b-6f50-4dab-9f5a-f3fbb5c153e5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.139405 Sep 18 06:24:48.053737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 24/45] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:24:46 2022] POST /compute/v2.1/servers => generated 401 bytes in 1148 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 06:24:48.060598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:48.061601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:48.069665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:48.070549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:48.083746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:48.084776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:48.090935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:48.091454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:48.108803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-05b0e1cd-e1ea-4708-8a6d-96ce5e8904ed tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.285747 Sep 18 06:24:48.110302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:24:46 2022] POST /compute/v2.1/servers => generated 401 bytes in 1288 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 06:24:48.115312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:48.116308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:48.150843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:48.175417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.025s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:48.175932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:48.183434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:48.184055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:48.187093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:48.238311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:48.255387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-753413f9-c8f1-4feb-9372-0044a7e06845 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1219 microversion: 2.1 time: 0.199738 Sep 18 06:24:48.256238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 25/47] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:48 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1219 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:48.261319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8ffd58b2-f7f9-4838-b683-2e3d90f2b366 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:48.261961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8ffd58b2-f7f9-4838-b683-2e3d90f2b366 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:48.489459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8ffd58b2-f7f9-4838-b683-2e3d90f2b366 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:48.497645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8ffd58b2-f7f9-4838-b683-2e3d90f2b366 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1237 microversion: 2.1 time: 0.240222 Sep 18 06:24:48.501386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 26/48] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:48 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1237 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:48.523205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2be5359b-4a31-44ff-a05c-f0b32aed7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:48.534951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2be5359b-4a31-44ff-a05c-f0b32aed7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:48.645242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2be5359b-4a31-44ff-a05c-f0b32aed7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:48.663510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2be5359b-4a31-44ff-a05c-f0b32aed7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1239 microversion: 2.1 time: 0.160789 Sep 18 06:24:48.667583 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 27/49] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:48 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1239 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:48.717857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:48.737276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b152f7db-d2b8-4c32-a34e-b1195e6bb6d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.625573 Sep 18 06:24:48.738129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 23/50] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:48 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:49.281434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:49.282048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:49.288133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:49.288617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:49.289057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:49.289749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:49.290202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:49.290711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:49.352683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:49.362854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fdafa370-120c-4ca2-ae82-15e22e268e58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1219 microversion: 2.1 time: 0.085242 Sep 18 06:24:49.363721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 28/51] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:49 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:49.524231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:49.524845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:49.531485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:49.531969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:49.532417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:49.533111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:49.533561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:49.534004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:49.595585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:49.603238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1294959f-dd31-41d1-ae4c-9135abde6b9d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1241 microversion: 2.1 time: 0.082981 Sep 18 06:24:49.603802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 24/52] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:49 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1241 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:49.691480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:49.692105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:49.698882 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:49.699501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:49.700024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:49.700806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:49.701327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:49.701762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:49.761144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:49.761854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:49.768235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:49.768742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:49.769194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:49.769893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:49.770448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:49.770968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:49.790437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:49.799087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a17aac97-2e43-4789-8230-64075520aad6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1243 microversion: 2.1 time: 0.111975 Sep 18 06:24:49.799825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 29/53] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:49 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1243 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:49.939211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:49.960735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bbc37bfc-2073-46c8-ab33-9f9c8df720a2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.203715 Sep 18 06:24:49.971083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 25/54] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:49 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:50.385620 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:50.386287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:50.392970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:50.393457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:50.393897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:50.394606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:50.395060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:50.395483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:50.454624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:50.462401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5ef346d6-eedb-4a79-a9c6-fed31399fc9b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1219 microversion: 2.1 time: 0.080993 Sep 18 06:24:50.462996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 30/55] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:50 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1219 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:50.627201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:50.627810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:50.633914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:50.634393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:50.634877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:50.635556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:50.636001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:50.636420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:50.695700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:50.703249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f86f3faf-1758-49e2-ba86-6de9efeedda8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1241 microversion: 2.1 time: 0.079839 Sep 18 06:24:50.703753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 26/56] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:50 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1241 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:50.822230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:50.822868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:50.829254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:50.829734 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:50.830163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:50.830874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:50.831363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:50.831814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:50.895083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:50.903864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-10d410d0-df6a-4cea-9ac6-34992fb794d9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1243 microversion: 2.1 time: 0.084958 Sep 18 06:24:50.904425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 31/57] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:50 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1243 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:50.984026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:50.984571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:50.990503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:50.991019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:50.991456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:50.992135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:50.992604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:50.993014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:51.120483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:51.128647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c927819b-eb2e-4f54-85c4-0130e0b6fbda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.148554 Sep 18 06:24:51.129203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 27/58] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:50 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:51.485323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:51.485977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:51.491758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:51.492234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:51.492665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:51.493332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:51.493778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:51.494194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:51.725897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:51.726539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:51.733030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:51.733519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:51.733955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:51.734703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:51.735328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:51.735756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:51.744463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:51.748510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:51.753828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d3cc4256-cc2e-4e51-82b6-644e5a2a93a6 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1219 microversion: 2.1 time: 0.272845 Sep 18 06:24:51.754370 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 32/59] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:51 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1219 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:51.798659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:51.806330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b7fbda4f-d9dd-4ee0-9386-b0d2c53f07a8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1241 microversion: 2.1 time: 0.084780 Sep 18 06:24:51.806999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 28/60] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:51 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1241 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:24:51.927045 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:51.927659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:51.934496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:51.935010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:51.935438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:51.936118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:51.936562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:51.936975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:51.997087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:52.006491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-db8250f3-35a9-45a8-945f-b31b35217be8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1235 microversion: 2.1 time: 0.084377 Sep 18 06:24:52.007331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 33/61] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:51 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1235 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:52.151335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:52.152242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:52.158522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:52.158933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:52.159561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:52.160251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:52.160871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:52.161291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:52.320796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:52.333417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-58c054b7-63f6-4a08-a4aa-89fef5740735 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.185765 Sep 18 06:24:52.334281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 29/62] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:52 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:52.783287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:52.784868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:52.793851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:52.794349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:52.795021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:52.795727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:52.796333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:52.796965 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:52.833560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:52.835884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:52.852714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:52.854332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:52.857731 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:52.861292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:52.863487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:52.864095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:53.004918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:53.013151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d38f4f8c-f83b-46c6-acc0-9c35f9a771d5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1211 microversion: 2.1 time: 0.238777 Sep 18 06:24:53.016961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:53.030552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 34/63] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:52 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1211 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:53.034063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bdea57d7-2a87-43cb-b0f8-fc79e81f62ed tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1233 microversion: 2.1 time: 0.207927 Sep 18 06:24:53.034642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 30/64] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:52 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1233 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:53.037253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:53.037868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:53.049189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:53.049871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:53.050479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:53.051269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:53.051860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:53.052485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:53.137896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:53.146755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bc55c162-a220-4eb7-90f2-4a2c1f8fa7ae tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1291 microversion: 2.1 time: 0.115029 Sep 18 06:24:53.147312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 35/65] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:53 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1291 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:53.359762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:53.360482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:53.366713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:53.367269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:53.367775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:53.368526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:53.369046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:53.369566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:53.511031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:53.522125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3deb0be2-e751-4852-bd83-714d8cb2ab15 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.166515 Sep 18 06:24:53.522891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 31/66] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:53 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:54.044790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:54.050066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:54.062908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:54.063526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:54.070454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:54.071276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:54.075603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:54.076561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:54.077421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:54.078491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:54.083078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:54.083585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:54.084033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:54.088388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:54.093299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:54.093755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:54.322548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:54.338865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7d317a64-2fdd-4d40-a5ac-bd84ad1c1911 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1211 microversion: 2.1 time: 0.304001 Sep 18 06:24:54.341204 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 36/67] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:54 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1211 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:54.354180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:54.354887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:54.382539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:54.385230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:54.385723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:54.388244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:54.391001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:54.393766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:54.539604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:54.558863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8461d70d-fc47-4001-bb18-86f1f8ae2177 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1233 microversion: 2.1 time: 0.496335 Sep 18 06:24:54.559499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 32/68] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:54 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1233 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:54.568380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:54.568999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:54.583014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:54.584050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:54.584662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:54.586016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:54.587417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:54.590796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:54.599616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:54.651301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:54.661730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4dc7f771-43ee-45e3-8d9f-048ae782a123 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1299 microversion: 2.1 time: 0.316903 Sep 18 06:24:54.670170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 37/69] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:54 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1299 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:55.117595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:55.139589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-056537f9-dd84-4a3c-ac90-af6da7a27e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.578504 Sep 18 06:24:55.142646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 33/70] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:54 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:55.371706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:55.373341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:55.393836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:55.394335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:55.394827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:55.395533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:55.395991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:55.396410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:55.479708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:55.490521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-facad5b3-20da-4c5d-bf12-6de49e878896 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1273 microversion: 2.1 time: 0.129623 Sep 18 06:24:55.491130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 38/71] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:55 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1273 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:55.585775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:55.586653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:55.599406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:55.601954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:55.604226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:55.605303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:55.607958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:55.608672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:55.691182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:55.692305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:55.703193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:55.705056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:55.705931 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:55.708255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:55.709540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:55.712915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:55.805451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:55.836251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e9ca8c5-1499-44b7-a45f-1aa921d227e6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1289 microversion: 2.1 time: 0.233728 Sep 18 06:24:55.836924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 34/72] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:55 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1289 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:24:55.917474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:55.928259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9bb41456-345a-4369-ba13-bd50ec3eab61 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1297 microversion: 2.1 time: 0.241670 Sep 18 06:24:55.932481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 39/73] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:55 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1297 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:56.164857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:56.166558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:56.179985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:56.180477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:56.180914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:56.181755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:56.193063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:56.195140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:56.515548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:56.516155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:56.528414 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:56.530844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:56.533186 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:56.537050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:56.537873 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:56.538500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:56.903614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:56.919050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c89baa7a-cd21-49f7-8a0f-b68dff18fa9e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1273 microversion: 2.1 time: 0.405715 Sep 18 06:24:56.921279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 40/74] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:56 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1273 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:24:56.937402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:56.938354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:56.955307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:56.958258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:56.959634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:56.963444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:56.964086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:56.967778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:57.427821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:57.472439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-527f62c7-eb85-477e-ab5f-5e049cc99958 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1295 microversion: 2.1 time: 0.540884 Sep 18 06:24:57.473184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 41/75] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:56 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1295 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:57.488692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:57.491171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:57.507217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:57.509558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:57.510676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:57.515017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:57.517134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:57.519126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:57.542129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:57.550419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d660139c-6b7e-41ab-bc8e-b467d863fbff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 1.390525 Sep 18 06:24:57.551569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 35/76] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:56 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 1392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:57.804947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-28067ca4-1055-41a0-99d0-0422f988e44a req-6e2f8866-845e-4a12-8002-cdb06162d63a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1dd95cf5-d6be-4de9-9941-8b1d35855bcb", "name": "network-vif-plugged", "status": "completed", "tag": "76758f98-a1a3-4d17-a73a-67726f56e2d5"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:57.815673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-28067ca4-1055-41a0-99d0-0422f988e44a req-6e2f8866-845e-4a12-8002-cdb06162d63a service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:57.816615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-28067ca4-1055-41a0-99d0-0422f988e44a req-6e2f8866-845e-4a12-8002-cdb06162d63a service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:57.817273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-28067ca4-1055-41a0-99d0-0422f988e44a req-6e2f8866-845e-4a12-8002-cdb06162d63a service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:57.834020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-28067ca4-1055-41a0-99d0-0422f988e44a req-6e2f8866-845e-4a12-8002-cdb06162d63a service nova] Creating event network-vif-plugged:76758f98-a1a3-4d17-a73a-67726f56e2d5 for instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb on hv-cinder-85535 Sep 18 06:24:57.840780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-28067ca4-1055-41a0-99d0-0422f988e44a req-6e2f8866-845e-4a12-8002-cdb06162d63a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.287253 Sep 18 06:24:57.847117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 36/77] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:24:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:57.932693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:57.952905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:57.954045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:57.957301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=6 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:57.976300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:57.977897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:57.979749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:57.982837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:57.986067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:57.987939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:57.989199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=14 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:58.035511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:58.054387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-13aaef97-b565-49ba-a534-fa74c0c0d94e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.571904 Sep 18 06:24:58.056445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 42/78] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:57 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:58.468460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:58.486841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ebf9ac8b-a563-4582-afeb-f4725f11b95a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.540728 Sep 18 06:24:58.487429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 37/79] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:57 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:24:58.511677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:58.512445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:58.523567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:58.524641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:58.525711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:58.528243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:58.529427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:58.531383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:58.573287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:58.575242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:58.588557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:58.589509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:58.591132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:58.595338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:58.597579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:58.598926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:58.657537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=18 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:58.716653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:58.728442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-056db22b-9640-4e9a-8c3e-4dbdde0060c4 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1295 microversion: 2.1 time: 0.222865 Sep 18 06:24:58.731839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 43/80] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:58 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1295 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:58.911951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-79f06676-61ad-4f69-875c-71e434718f1a req-66b0d1c3-6ea9-44c9-b724-2810b194f1ff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1d1894f2-f7ef-434c-8f30-0f99e862fd23", "tag": "b4a2f92b-82f0-498d-b500-c9a5d54ef49d"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:58.937451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-79f06676-61ad-4f69-875c-71e434718f1a req-66b0d1c3-6ea9-44c9-b724-2810b194f1ff service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:58.939360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-79f06676-61ad-4f69-875c-71e434718f1a req-66b0d1c3-6ea9-44c9-b724-2810b194f1ff service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:58.940654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-79f06676-61ad-4f69-875c-71e434718f1a req-66b0d1c3-6ea9-44c9-b724-2810b194f1ff service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:58.964324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-79f06676-61ad-4f69-875c-71e434718f1a req-66b0d1c3-6ea9-44c9-b724-2810b194f1ff service nova] Creating event network-changed:b4a2f92b-82f0-498d-b500-c9a5d54ef49d for instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 on hv-cinder-85535 Sep 18 06:24:58.974363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-79f06676-61ad-4f69-875c-71e434718f1a req-66b0d1c3-6ea9-44c9-b724-2810b194f1ff service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070592 Sep 18 06:24:58.975820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 44/81] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:24:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:24:59.081031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:59.081743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:59.107013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:59.107846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:59.109083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:59.109953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:59.110515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:59.111165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:59.129879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:59.149356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-60234b71-38ad-42a1-b9cf-146dbad9c7a0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.579117 Sep 18 06:24:59.150169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 38/82] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:58 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:59.397663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:59.409229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-31868147-b59f-4421-b7c9-905229b9a880 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.332591 Sep 18 06:24:59.410121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 45/83] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:59 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:24:59.487598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-b15c30d0-b328-4f88-8270-9ff921e007de req-0bc9d10f-d062-4b22-bc5c-52b3a7465e54 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb5ac07d-bc0f-455d-a133-6abb2e30f695", "tag": "72dd4c69-429a-42fe-8a60-ffe8316cc094"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:24:59.497447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-b15c30d0-b328-4f88-8270-9ff921e007de req-0bc9d10f-d062-4b22-bc5c-52b3a7465e54 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:59.504800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-b15c30d0-b328-4f88-8270-9ff921e007de req-0bc9d10f-d062-4b22-bc5c-52b3a7465e54 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:59.505509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-b15c30d0-b328-4f88-8270-9ff921e007de req-0bc9d10f-d062-4b22-bc5c-52b3a7465e54 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:59.518492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:59.519373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:59.529852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:59.530485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-b15c30d0-b328-4f88-8270-9ff921e007de req-0bc9d10f-d062-4b22-bc5c-52b3a7465e54 service nova] Creating event network-changed:72dd4c69-429a-42fe-8a60-ffe8316cc094 for instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 on hv-cinder-85535 Sep 18 06:24:59.530632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:59.535037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:59.536224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:59.539072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:59.539753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:59.540369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-b15c30d0-b328-4f88-8270-9ff921e007de req-0bc9d10f-d062-4b22-bc5c-52b3a7465e54 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056379 Sep 18 06:24:59.541000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 39/84] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:24:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:24:59.700347 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:59.709692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-76c827f5-7bb6-4c9e-93c1-0d1f32757f8c tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.196828 Sep 18 06:24:59.725543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 46/85] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:59 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:24:59.753975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:24:59.755660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:24:59.762846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:59.763495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:59.764016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:59.764751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:24:59.765221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:24:59.765815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:24:59.934287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:24:59.944112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b6f61231-666b-4868-8375-2545225bb85f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.194554 Sep 18 06:24:59.944741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 40/86] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:24:59 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:00.018754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-bd4ddace-4be2-48be-aa9d-08f6fb166c2e req-0467cc76-6f3b-42c9-b96b-fdf58161808f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d8351557-8730-42b4-9788-684192b14e98", "name": "network-vif-plugged", "status": "completed", "tag": "b4df9936-75d5-4973-a602-6e13994ffd2a"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:00.029849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-bd4ddace-4be2-48be-aa9d-08f6fb166c2e req-0467cc76-6f3b-42c9-b96b-fdf58161808f service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.030718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-bd4ddace-4be2-48be-aa9d-08f6fb166c2e req-0467cc76-6f3b-42c9-b96b-fdf58161808f service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.032038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-bd4ddace-4be2-48be-aa9d-08f6fb166c2e req-0467cc76-6f3b-42c9-b96b-fdf58161808f service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:00.048458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-bd4ddace-4be2-48be-aa9d-08f6fb166c2e req-0467cc76-6f3b-42c9-b96b-fdf58161808f service nova] Creating event network-vif-plugged:b4df9936-75d5-4973-a602-6e13994ffd2a for instance d8351557-8730-42b4-9788-684192b14e98 on hv-cinder-85535 Sep 18 06:25:00.053919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-bd4ddace-4be2-48be-aa9d-08f6fb166c2e req-0467cc76-6f3b-42c9-b96b-fdf58161808f service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038939 Sep 18 06:25:00.056018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 47/87] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:25:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:25:00.173257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:00.173920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:00.180844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.181694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.182341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:00.183148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.184109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.184828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:00.349863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:00.358243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-effb298f-0260-4ef0-a23d-07af186eab29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.188892 Sep 18 06:25:00.358851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 41/88] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:00 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:00.434383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:00.435364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:00.442042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.442536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.443007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:00.443706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.444211 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.444573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:00.613472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:00.622404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8c875fa2-7b30-44ee-b10f-01284576f97b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.192736 Sep 18 06:25:00.622991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 48/89] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:00 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:00.734456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:00.736049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:00.744314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.745328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.746811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:00.751292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.752127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.753164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:00.970300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:00.971642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:00.975741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:00.986357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.992724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.993379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:00.994379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:00.995067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:00.995512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:01.246715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:01.266690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5a590520-5c60-4b10-a7dc-54534e5e77fa tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.537955 Sep 18 06:25:01.267338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 42/90] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:00 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:01.381602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:01.386359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:01.388093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:01.396390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-31dc0e27-9413-4f76-9644-977dc30f8cb5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.432180 Sep 18 06:25:01.397991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 49/91] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:00 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:01.418999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:01.421801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:01.425910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:01.427692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:01.428404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:01.429551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:01.648011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:01.648780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:01.655765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:01.658328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:01.659165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:01.659875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:01.660336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:01.660775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:02.024061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:02.035562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e0f914e2-81da-48b3-8493-3aa3160f4484 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.654914 Sep 18 06:25:02.050140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 43/92] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:01 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:02.167724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:02.172478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:02.179913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-aedab194-beed-4537-b1f7-6ebbb1279f9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.536121 Sep 18 06:25:02.180589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 50/93] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:01 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:02.288120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-63698315-8ea6-43e7-a822-aa88ec1a823e req-934cb0b8-0261-4913-8ece-d2db85ce82ed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "045c9108-b1cd-4720-94bc-7aa9159c26d4", "tag": "9ebd90c7-97ce-45a5-8838-7c819266affb"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:02.290846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:02.291459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:02.299092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-63698315-8ea6-43e7-a822-aa88ec1a823e req-934cb0b8-0261-4913-8ece-d2db85ce82ed service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:02.299560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-63698315-8ea6-43e7-a822-aa88ec1a823e req-934cb0b8-0261-4913-8ece-d2db85ce82ed service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:02.300022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-63698315-8ea6-43e7-a822-aa88ec1a823e req-934cb0b8-0261-4913-8ece-d2db85ce82ed service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:02.300444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:02.301158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:02.301370 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:02.302062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:02.303017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:02.303450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:02.316955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-63698315-8ea6-43e7-a822-aa88ec1a823e req-934cb0b8-0261-4913-8ece-d2db85ce82ed service nova] Creating event network-changed:9ebd90c7-97ce-45a5-8838-7c819266affb for instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 on hv-cinder-85535 Sep 18 06:25:02.323752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-63698315-8ea6-43e7-a822-aa88ec1a823e req-934cb0b8-0261-4913-8ece-d2db85ce82ed service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042806 Sep 18 06:25:02.324959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 44/94] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:25:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:25:02.426598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:02.427575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:02.435722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:02.436240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:02.436730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:02.437495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:02.438085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:02.438514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:02.464372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:02.475648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8b2b4299-8a1d-49c7-90eb-d38c75ceef9a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.188532 Sep 18 06:25:02.476445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 51/95] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:02 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:02.580970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:02.589269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-012ce7d2-41c6-49c6-a3c7-1b6400e6180d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.168514 Sep 18 06:25:02.590619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 45/96] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:02 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:03.061794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:03.062637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:03.068470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:03.069025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:03.069539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:03.070277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:03.070846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:03.071335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:03.202437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:03.203159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:03.209517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:03.210281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:03.210943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:03.211647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:03.212076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:03.212691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:03.218115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:03.226496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3e64df01-9ab8-40a8-a2b8-d0cbb78ab92f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.168961 Sep 18 06:25:03.227092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 52/97] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:03 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:03.350034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:03.358143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7ff4784f-d4fd-46f4-9167-4b5e92cdd2dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.159742 Sep 18 06:25:03.358729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 46/98] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:03 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:03.499786 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:03.500398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:03.506373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:03.506907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:03.507326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:03.508007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:03.508731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:03.509151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:03.613690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:03.614374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:03.621492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:03.622018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:03.622459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:03.624094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:03.624563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:03.627728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:03.949732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:03.959033 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-694f0f1c-d4e9-47aa-90f3-e567d76f540a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.463512 Sep 18 06:25:03.959635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 53/99] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:03 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:04.072480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:04.083311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-58617903-3043-4bc7-86ca-1656a5f7d21d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.473419 Sep 18 06:25:04.084446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 47/100] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:03 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:04.256510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:04.257157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:04.267761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:04.271031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:04.271509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:04.272657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:04.273084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:04.273509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:04.387081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:04.388527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:04.403544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:04.405626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:04.406656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:04.408900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:04.410132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:04.413036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:04.916955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:04.938163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a99d477d-9d25-47da-9c58-8d99a2555590 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1322 microversion: 2.1 time: 0.685968 Sep 18 06:25:04.970899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 54/101] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:04 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1322 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:04.983682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:04.988683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:04.989991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:04.995376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3c1fb8f6-12e2-45b5-8717-ff38f4a885ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.615307 Sep 18 06:25:04.996481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 48/102] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:04 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:05.004032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:05.004897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:05.005539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:05.006667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:05.007375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:05.010988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:05.107539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:05.108169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:05.121400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:05.121960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:05.122348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:05.127874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:05.128296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:05.128747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:05.536096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:05.545847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9ff3ec49-6578-4882-b76e-b4f5d6db7b17 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.567221 Sep 18 06:25:05.546428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 55/103] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:04 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:05.573648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:05.583119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ee29e2fa-9bef-4c18-b995-1a2d04ec1307 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.480165 Sep 18 06:25:05.583698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 49/104] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:05 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:05.967358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:05.967963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:05.973895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:05.974372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:05.974844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:05.975525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:05.975970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:05.976391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.025273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:06.026065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:06.032712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.033471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.033906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.034986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.035153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.035791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.162308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:06.171907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0648cd08-dcf4-4035-9797-a362fe21d321 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1515 microversion: 2.1 time: 0.208600 Sep 18 06:25:06.172679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 56/105] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:05 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1515 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:06.195721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:06.197941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:06.204210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.204495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.204928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.205629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.206098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.206542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.207691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:06.218621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4117f646-5ca4-41cf-bd28-b1d55276e46b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.197277 Sep 18 06:25:06.219452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 50/106] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:06 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:06.276832 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lazy-loading 'flavor' on Instance uuid d8351557-8730-42b4-9788-684192b14e98 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:25:06.454521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2fe09d4e-68d1-41c6-b543-265ae51dacaf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98/action" status: 200 len: 32 microversion: 2.1 time: 0.262149 Sep 18 06:25:06.455320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 57/107] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:25:06 2022] POST /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98/action => generated 32 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 18 06:25:06.475031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:06.476012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:06.484776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.485724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.486364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.487287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.487885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.488546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.570152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:06.571400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:06.579810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.580860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.581478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.582565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.583250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.584017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.674290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:06.680523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:06.688171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8d504beb-bda9-4d4b-b2ce-9c596cd03649 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.219603 Sep 18 06:25:06.688993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 51/108] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:06 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:06.696789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:06.699497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:06.711496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.712531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.713213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:06.714654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:06.716078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:06.716937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:07.045371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:07.053304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f7c9eddb-cec6-4e1b-83c2-fa8c9335d1d8 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.487534 Sep 18 06:25:07.054447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 58/109] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:06 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:07.060142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:07.069189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-350169f9-81ac-4d11-8704-a14d9fcb3247 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.378143 Sep 18 06:25:07.069873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 52/110] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:06 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:07.245373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:07.246286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:07.252329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:07.252992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:07.253436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:07.254207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:07.254733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:07.255388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:07.422557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:07.431727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5dc2cd36-6278-41f5-bd5d-1ae8b6510498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.190230 Sep 18 06:25:07.432321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 59/111] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:07 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:07.718195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:07.718867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:07.725146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:07.725655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:07.726096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:07.726829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:07.727351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:07.728084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:07.869615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:07.877777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0a6d0a1c-eb92-4b77-99fc-a0e84317a45d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.165808 Sep 18 06:25:07.878345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 53/112] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:07 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:08.076110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:08.076711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:08.082857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:08.083357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:08.083786 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:08.084480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:08.084929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:08.085345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:08.091983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:08.093285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:08.100453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:08.101028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:08.101539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:08.102474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:08.103139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:08.103798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:08.312330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:08.315515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:08.320401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-154e0c2c-b878-436e-b348-77b67a8f12e5 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.248435 Sep 18 06:25:08.320965 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 60/113] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:08 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:08.323128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d6e4b2d0-7d8e-4a14-9457-9f3052099b13 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.234943 Sep 18 06:25:08.323690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 54/114] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:08 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:08.471251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:08.471879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:08.477984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:08.478488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:08.478950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:08.479624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:08.480094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:08.480511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:08.613603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:08.621954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9133352b-00f4-45b2-bbd4-84cd3e52af0d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.154532 Sep 18 06:25:08.622524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 61/115] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:08 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:08.902499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:08.903221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:08.909022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:08.909500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:08.909929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:08.910643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:08.911113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:08.911685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:09.068949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:09.079731 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7b1a7578-e360-4988-b814-3a5fd4cc4598 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.180958 Sep 18 06:25:09.094603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 55/116] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:08 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:09.343616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:09.344797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:09.347270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:09.349148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:09.353009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:09.353565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:09.354005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:09.354905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:09.355863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:09.356640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:09.361803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:09.362341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:09.362895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:09.363615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:09.364062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:09.364465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:09.724743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:09.745486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-65af30c9-365e-4bee-8087-a1253e44bc0a tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.405952 Sep 18 06:25:09.746312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 62/117] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:09 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:09.751386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:09.751996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:09.758396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:09.763214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:09.763840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:09.764783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:09.765236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:09.765658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:09.819504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:09.828695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-00802d0b-9efb-4b08-a6cf-6e110c0cf02b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.486826 Sep 18 06:25:09.841361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 56/118] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:09 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:10.066687 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:10.081110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8cd972f5-43d6-48bb-b3d0-31b3e6f5f643 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.332974 Sep 18 06:25:10.084601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 63/119] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:09 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:10.112760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:10.114273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:10.127408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:10.127894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:10.137725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:10.138398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:10.138916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:10.139348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:10.441314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:10.476849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4c78a009-3aac-4c7c-8237-6b30938e9e1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.371093 Sep 18 06:25:10.478828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 57/120] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:10 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:10.769229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:10.770001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:10.786061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:10.794852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:10.797467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:10.801533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:10.803552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:10.804047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:10.858061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:10.859423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:10.870334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:10.871655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:10.872233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:10.873148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:10.873733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:10.875612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:11.061963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:11.075490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9893c1b3-fe39-4fd5-bdb4-c1d4dce5e400 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.309616 Sep 18 06:25:11.082872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 64/121] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:10 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:11.112739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:11.124817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f3fb9118-6fc8-4b50-bc65-80e9fa69a51e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.271252 Sep 18 06:25:11.133987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:11.134274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 58/122] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:10 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:11.134667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:11.143063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:11.143696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:11.144331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:11.145805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:11.146454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:11.147225 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:11.175479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:11.314827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:11.325427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8db54d69-ecb3-40f8-a87f-9fe7707adf54 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.208568 Sep 18 06:25:11.326110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 65/123] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:11 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:11.504983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:11.505784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:11.524385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:11.524917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:11.525365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:11.526076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:11.529510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:11.529955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:11.781925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:11.797182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d57928f3-35f2-46a9-9e17-6ab3b5f8259b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.298104 Sep 18 06:25:11.801716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 59/124] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:11 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:12.102312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:12.103270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:12.113660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.114384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.116255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:12.117128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.119506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.120183 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:12.148124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:12.148737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:12.165204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.165756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.166200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:12.166954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.167539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.167962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:12.460943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:12.482885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0a5ccce1-29da-4206-ae24-c24117221620 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.338326 Sep 18 06:25:12.484296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 60/125] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:12 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:12.502535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:12.516533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:12.517157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:12.521051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d30a8879-07f7-4ff5-8b7e-821d20aad19b tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.424581 Sep 18 06:25:12.527294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 66/126] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:12 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:12.529343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.529885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.530286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:12.531068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.531562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.531927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:12.550856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-64b86191-0ab4-4425-9eb1-f6f8e0d08008 req-db7cf371-2963-4327-86f2-083deaf705ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1d1894f2-f7ef-434c-8f30-0f99e862fd23", "name": "network-vif-plugged", "status": "completed", "tag": "b4a2f92b-82f0-498d-b500-c9a5d54ef49d"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:12.581242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-64b86191-0ab4-4425-9eb1-f6f8e0d08008 req-db7cf371-2963-4327-86f2-083deaf705ba service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.581821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-64b86191-0ab4-4425-9eb1-f6f8e0d08008 req-db7cf371-2963-4327-86f2-083deaf705ba service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.593319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-64b86191-0ab4-4425-9eb1-f6f8e0d08008 req-db7cf371-2963-4327-86f2-083deaf705ba service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:12.616032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-64b86191-0ab4-4425-9eb1-f6f8e0d08008 req-db7cf371-2963-4327-86f2-083deaf705ba service nova] Creating event network-vif-plugged:b4a2f92b-82f0-498d-b500-c9a5d54ef49d for instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 on hv-cinder-85535 Sep 18 06:25:12.630454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-64b86191-0ab4-4425-9eb1-f6f8e0d08008 req-db7cf371-2963-4327-86f2-083deaf705ba service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086762 Sep 18 06:25:12.639425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 67/127] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:25:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:25:12.823864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:12.832806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:12.833527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:12.849445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0a78c2a-8294-407e-b1d2-ed30e9e987df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.360525 Sep 18 06:25:12.850051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 61/128] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:12 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:12.853793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.854311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.855054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:12.855937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:12.856445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:12.857084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:13.344632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:13.356733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8c93e1cb-ee2d-4e11-8250-9b7165b689b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.536849 Sep 18 06:25:13.358315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 68/129] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:12 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:13.508919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:13.510176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:13.516141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:13.517211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:13.517733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:13.518542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:13.519080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:13.519704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:13.556251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:13.557788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:13.577286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:13.578454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:13.583417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:13.586307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:13.587173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:13.588476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:13.732027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:13.748000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8568a4f4-951d-4ec4-88ba-205ffe8c639f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.244015 Sep 18 06:25:13.749078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 62/130] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:13 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:13.791851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:13.804824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3bd92ae4-f311-4730-bc14-2c2aba713c23 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.256534 Sep 18 06:25:13.816455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 69/131] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:13 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:13.883757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:13.884275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:13.887734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:13.893539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:13.894028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:13.894457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:13.895363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:13.895818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:13.896256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:14.098820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:14.114416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4d0732c3-bce9-44fc-be00-7f8f8712cac9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.239487 Sep 18 06:25:14.117545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 63/132] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:13 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:14.382951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:14.383983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:14.395091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:14.395711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:14.396406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:14.399666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:14.400149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:14.400766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:14.604962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:14.614324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c4dd6523-af60-400d-8062-1507b3f54ed1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.235245 Sep 18 06:25:14.615221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 70/133] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:14 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:14.684103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-6573a254-a17f-4473-9cf0-f014ffc6a5dd req-caa0ea08-322d-4061-9644-a2e241772e17 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb5ac07d-bc0f-455d-a133-6abb2e30f695", "name": "network-vif-plugged", "status": "completed", "tag": "72dd4c69-429a-42fe-8a60-ffe8316cc094"}, {"server_uuid": "045c9108-b1cd-4720-94bc-7aa9159c26d4", "name": "network-vif-plugged", "status": "completed", "tag": "9ebd90c7-97ce-45a5-8838-7c819266affb"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:14.710321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6573a254-a17f-4473-9cf0-f014ffc6a5dd req-caa0ea08-322d-4061-9644-a2e241772e17 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:14.710840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6573a254-a17f-4473-9cf0-f014ffc6a5dd req-caa0ea08-322d-4061-9644-a2e241772e17 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:14.711470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6573a254-a17f-4473-9cf0-f014ffc6a5dd req-caa0ea08-322d-4061-9644-a2e241772e17 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:14.732747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-6573a254-a17f-4473-9cf0-f014ffc6a5dd req-caa0ea08-322d-4061-9644-a2e241772e17 service nova] Creating event network-vif-plugged:72dd4c69-429a-42fe-8a60-ffe8316cc094 for instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 on hv-cinder-85535 Sep 18 06:25:14.733544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-6573a254-a17f-4473-9cf0-f014ffc6a5dd req-caa0ea08-322d-4061-9644-a2e241772e17 service nova] Creating event network-vif-plugged:9ebd90c7-97ce-45a5-8838-7c819266affb for instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 on hv-cinder-85535 Sep 18 06:25:14.745673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-6573a254-a17f-4473-9cf0-f014ffc6a5dd req-caa0ea08-322d-4061-9644-a2e241772e17 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.067841 Sep 18 06:25:14.746654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 64/134] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:25:14 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:25:14.773942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:14.774555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:14.787316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:14.788481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:14.789390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:14.790744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:14.792961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:14.794065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:14.832166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:14.835075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:14.846660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:14.851713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:14.852779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:14.854443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:14.856006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:14.856681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:15.301104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:15.310041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e5616951-0585-4269-ba92-3b8a5c3ddd24 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.542230 Sep 18 06:25:15.312099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 71/135] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:14 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:15.317568 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:15.318354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:15.325380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:15.326284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:15.326930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:15.328123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:15.329185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:15.329774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:15.372245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:15.390639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2d629836-f2c2-4dab-bddd-3d05188fe2dc tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1315 microversion: 2.1 time: 0.560262 Sep 18 06:25:15.395201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 65/136] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:14 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1315 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:15.568535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:15.577431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-15267cdd-24e4-4ad3-819e-a7f6244056c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.264070 Sep 18 06:25:15.578192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 72/137] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:15 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:15.640870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:15.643278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:15.655081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:15.656023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:15.657034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:15.658880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:15.659586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:15.660230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:15.882770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:15.891315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-129d790f-d797-43a4-8ffe-d3ef8364645b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.256296 Sep 18 06:25:15.892207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 66/138] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:15 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:16.338170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:16.338818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:16.345005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:16.345477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:16.346094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:16.346950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:16.348906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:16.349345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:16.420727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:16.421340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:16.427388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:16.428175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:16.428860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:16.429714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:16.430322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:16.430943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:16.603198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:16.613338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-49be6dd1-fbad-415d-aed2-9c82f8a38f7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1362 microversion: 2.1 time: 0.279096 Sep 18 06:25:16.626419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 73/139] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:16 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1362 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:16.636999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:16.637830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:16.644077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:16.644808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:16.645318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:16.646196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:16.646875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:16.647457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:16.698709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:16.712232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e28e4655-9a2c-4ede-bd29-a50bf776eeb2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1502 microversion: 2.1 time: 0.298494 Sep 18 06:25:16.712816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 67/140] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:16 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1502 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:16.863119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:16.875193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7d208bf9-736a-454c-9065-a6b129250b5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1364 microversion: 2.1 time: 0.246920 Sep 18 06:25:16.876876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 74/141] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:16 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1364 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:16.915368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:16.922208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:16.931504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:16.933617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:16.937205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:16.938499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:16.939290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:16.939915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:17.108686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:17.117814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cd528c1b-bfdf-48a5-8d2f-5912329badf7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.206643 Sep 18 06:25:17.118493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 68/142] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:16 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:17.643079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:17.644334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:17.652637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:17.653360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:17.654029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:17.655170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:17.655955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:17.656690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:17.834407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:17.843742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3b84a801-af95-428b-a04d-1ef3e9fc27cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1547 microversion: 2.1 time: 0.205712 Sep 18 06:25:17.844518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 75/143] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:17 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1547 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:17.867865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:17.868368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:17.875009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:17.875988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:17.876437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:17.877329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:17.877773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:17.878458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:17.911748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:17.912520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:17.913360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:17.919926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:17.920675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:17.921288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:17.922025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:17.922478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:17.922930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:18.050964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1437e0af-c4c6-47cd-921b-2b8d7df0ad8c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.186924 Sep 18 06:25:18.051865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 69/144] 10.222.0.20 () {58 vars in 1275 bytes} [Sun Sep 18 06:25:17 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/os-interface => generated 284 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:25:18.139713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:18.143278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:18.143788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:18.150995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-427472d1-cc18-4d3f-bc4d-651bcb8dcf55 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 200 len: 1554 microversion: 2.1 time: 0.244563 Sep 18 06:25:18.152142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 76/145] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:17 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 1554 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:18.154338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:18.157383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:18.157850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:18.160766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:18.161277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:18.161866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:18.174476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:18.175132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:18.185789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:18.186374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:18.187659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:18.196288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:18.197689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:18.199080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:18.454744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-28b072d3-0672-4111-9a0e-7981a0147d9d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.284297 Sep 18 06:25:18.458717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 77/146] 10.222.0.20 () {58 vars in 1275 bytes} [Sun Sep 18 06:25:18 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-interface => generated 284 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:25:18.466630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e8510cdf-7f38-4d7d-9de3-bf678978d08c"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:18.469046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:18.483577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:18.489354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:18.490260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:18.492763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:18.494119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:18.494815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:18.519637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:18.536939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-841872c1-3b58-4549-bbfb-3df248dcfdf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.398780 Sep 18 06:25:18.548205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 70/147] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:18 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:18.969007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-17935d50-9d09-48f3-aebb-d35d8b6a788d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "POST /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.508190 Sep 18 06:25:18.970526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 78/148] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:25:18 2022] POST /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23/os-volume_attachments => generated 194 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:25:19.566491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:19.567836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:19.575254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:19.576237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:19.576765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:19.577481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:19.578100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:19.578932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:19.728362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:19.737942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bb26cbb4-cb87-4c01-9276-d4927d3949be tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.176214 Sep 18 06:25:19.738709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 71/149] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:19 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:20.746779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-a50e886c-8249-4d88-9a0a-f0563eb8878d req-38dcc767-3582-4bd3-9111-324e2085bb33 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "045c9108-b1cd-4720-94bc-7aa9159c26d4", "tag": "9ebd90c7-97ce-45a5-8838-7c819266affb"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:20.761774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:20.762616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:20.770876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a50e886c-8249-4d88-9a0a-f0563eb8878d req-38dcc767-3582-4bd3-9111-324e2085bb33 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:20.772474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:20.773055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:20.773579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:20.774365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:20.774940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a50e886c-8249-4d88-9a0a-f0563eb8878d req-38dcc767-3582-4bd3-9111-324e2085bb33 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:20.775596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a50e886c-8249-4d88-9a0a-f0563eb8878d req-38dcc767-3582-4bd3-9111-324e2085bb33 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:20.776251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:20.779794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:20.793765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-a50e886c-8249-4d88-9a0a-f0563eb8878d req-38dcc767-3582-4bd3-9111-324e2085bb33 service nova] Creating event network-changed:9ebd90c7-97ce-45a5-8838-7c819266affb for instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 on hv-cinder-85535 Sep 18 06:25:20.799123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-a50e886c-8249-4d88-9a0a-f0563eb8878d req-38dcc767-3582-4bd3-9111-324e2085bb33 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057598 Sep 18 06:25:20.800092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 79/150] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:25:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:25:20.981696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:20.997381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f9489ff5-43d1-4d43-9465-225f9d44dc8c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.239518 Sep 18 06:25:20.998955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 72/151] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:20 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:22.028562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:22.030386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:22.041876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:22.042864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:22.043761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:22.044872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:22.045489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:22.046158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:22.254257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:22.267814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-294f9ad5-ce7a-490c-8710-dc462391941b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.248593 Sep 18 06:25:22.268922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 80/152] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:22 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:22.813804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-746fa840-ed44-439b-b055-3b5576cd98ae req-e48f9cf5-af86-47d2-a10a-6c1de59209ae service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb5ac07d-bc0f-455d-a133-6abb2e30f695", "tag": "72dd4c69-429a-42fe-8a60-ffe8316cc094"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:22.823332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-746fa840-ed44-439b-b055-3b5576cd98ae req-e48f9cf5-af86-47d2-a10a-6c1de59209ae service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:22.823800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-746fa840-ed44-439b-b055-3b5576cd98ae req-e48f9cf5-af86-47d2-a10a-6c1de59209ae service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:22.824224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-746fa840-ed44-439b-b055-3b5576cd98ae req-e48f9cf5-af86-47d2-a10a-6c1de59209ae service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:22.835255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-746fa840-ed44-439b-b055-3b5576cd98ae req-e48f9cf5-af86-47d2-a10a-6c1de59209ae service nova] Creating event network-changed:72dd4c69-429a-42fe-8a60-ffe8316cc094 for instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 on hv-cinder-85535 Sep 18 06:25:22.840963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-746fa840-ed44-439b-b055-3b5576cd98ae req-e48f9cf5-af86-47d2-a10a-6c1de59209ae service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030811 Sep 18 06:25:22.841541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 73/153] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:25:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:25:23.291856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:23.293391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:23.300306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:23.300863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:23.301366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:23.302125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:23.302733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:23.303229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:23.682744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:23.690446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-328a72bc-4330-4f80-bbee-c978d514f7d2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.403161 Sep 18 06:25:23.691479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 81/154] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:23 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:24.715117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:24.715963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:24.721988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:24.722464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:24.722934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:24.723610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:24.724055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:24.724470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:24.873680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:24.881488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6d2ee791-537b-4001-8c29-762ba1be50ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.170548 Sep 18 06:25:24.882256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 74/155] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:24 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:25.905667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:25.906288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:25.912983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:25.913476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:25.913916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:25.914636 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:25.915094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:25.915525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:25.934596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:26.048408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:26.056231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-de39a8f1-0339-4c3b-8b6d-2dd429f0c58c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.154939 Sep 18 06:25:26.056789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 82/156] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:25 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:27.079566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:27.080190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:27.087504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:27.091264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:27.094450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:27.095477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:27.095988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:27.096426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:27.255453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:27.263799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5bff9305-b21e-475d-ba37-f6edd4e729c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.188291 Sep 18 06:25:27.264368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 75/157] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:27 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:28.287583 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:28.288197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:28.294805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:28.295286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:28.295711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:28.296390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:28.296834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:28.297252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:28.478047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:28.489002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6971441d-e2a0-4f75-a945-479023dbb9a9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.205156 Sep 18 06:25:28.489743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 83/158] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:28 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:28.787357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:28.787857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:28.794605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:28.795101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:28.795535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:28.796216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:28.796665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:28.797083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:28.822142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:25:28.929924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b72816e9-9f92-42fc-b7bf-79f3f8bd0ba7 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 204 len: 0 microversion: 2.1 time: 0.146288 Sep 18 06:25:28.930632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 76/159] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:25:28 2022] DELETE /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:25:28.939957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:28.940718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:28.949395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:28.949875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:28.950310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:28.951018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:28.951465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:28.951878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:29.099290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:29.107779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5d2e32ef-7574-417f-a056-de7e63a66698 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1554 microversion: 2.1 time: 0.171908 Sep 18 06:25:29.108340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 84/160] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:28 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1554 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:29.515374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:29.516734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:29.522883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:29.523364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:29.523801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:29.524477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:29.524923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:29.525346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:29.658646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:29.667459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c4b975fa-2eb7-4d0f-be44-56abb67a29f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.157830 Sep 18 06:25:29.668180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 77/161] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:29 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:30.130048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:30.130801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:30.137622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:30.138105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:30.138549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:30.139297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:30.139752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:30.140174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:30.282184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:30.290759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e40007e9-bfbe-47c7-9c61-f8637f8371ec tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1554 microversion: 2.1 time: 0.164535 Sep 18 06:25:30.302617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 85/162] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:30 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1554 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:30.691281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:30.692054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:30.700088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:30.700088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:30.700088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:30.700088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:30.700539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:30.700965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:30.884313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:30.891973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e060cc60-fbc1-4f02-8c79-326688395f6c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.205022 Sep 18 06:25:30.892658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 78/163] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:30 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:31.313836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:31.314450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:31.316806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=73 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:31.321985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:31.322989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:31.323431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:31.324115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:31.324566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:31.324998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:31.470000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:31.482591 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e4f61e6a-b2a5-4b78-999d-55dbd715ed58 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1554 microversion: 2.1 time: 0.172370 Sep 18 06:25:31.484091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 86/164] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:31 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1554 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:31.916791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:31.918287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:31.924900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:31.925631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:31.926134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:31.926938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:31.927571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:31.928200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:32.100276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:32.108860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e5ee8a16-cb4f-42b2-8a2c-e0d61315b1b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.196376 Sep 18 06:25:32.109443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 79/165] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:31 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:32.506855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:32.507471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:32.514892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:32.515381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:32.515822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:32.516515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:32.516972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:32.517397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:32.664904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:32.674276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a52937a6-4442-454a-a87a-c72ee12a93c2 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1554 microversion: 2.1 time: 0.171357 Sep 18 06:25:32.674900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 87/166] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:32 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1554 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:33.133137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:33.133732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:33.142604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:33.143293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:33.144038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:33.144906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:33.145703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:33.146131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:33.613575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:33.625770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-17e2d1cf-c8f7-4ea0-a366-1301ba04545c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.493695 Sep 18 06:25:33.639143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 80/167] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:33 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:33.700097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:33.701455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:33.711927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:33.717136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:33.719279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:33.721040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:33.722336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:33.724438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:34.029676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:34.043197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-62c3e8a3-5062-4e75-8ef1-135e2460616e tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1554 microversion: 2.1 time: 0.348384 Sep 18 06:25:34.043850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 88/168] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:33 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1554 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:34.662825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:34.662825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:34.669809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:34.670320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:34.670789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:34.671982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:34.671982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:34.672424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:34.992416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:35.010271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cf87ac4e-42d5-4194-9e8b-07c1a2d98ee8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.354928 Sep 18 06:25:35.012323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 81/169] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:34 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:35.065408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:35.066751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:35.081964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:35.086956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:35.087701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:35.092555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:35.093250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:35.096298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:35.130860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-e35f1ae0-3c8a-4c2e-96be-8adf1ecb4e92 req-672997a4-ff3f-4431-8e52-5669e69539e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1d1894f2-f7ef-434c-8f30-0f99e862fd23", "name": "network-vif-deleted", "tag": "b4a2f92b-82f0-498d-b500-c9a5d54ef49d"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:35.147256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-e35f1ae0-3c8a-4c2e-96be-8adf1ecb4e92 req-672997a4-ff3f-4431-8e52-5669e69539e8 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:35.148072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-e35f1ae0-3c8a-4c2e-96be-8adf1ecb4e92 req-672997a4-ff3f-4431-8e52-5669e69539e8 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:35.148675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-e35f1ae0-3c8a-4c2e-96be-8adf1ecb4e92 req-672997a4-ff3f-4431-8e52-5669e69539e8 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:35.169249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-e35f1ae0-3c8a-4c2e-96be-8adf1ecb4e92 req-672997a4-ff3f-4431-8e52-5669e69539e8 service nova] Creating event network-vif-deleted:b4a2f92b-82f0-498d-b500-c9a5d54ef49d for instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 on hv-cinder-85535 Sep 18 06:25:35.187553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-e35f1ae0-3c8a-4c2e-96be-8adf1ecb4e92 req-672997a4-ff3f-4431-8e52-5669e69539e8 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059711 Sep 18 06:25:35.189062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 82/170] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:25:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:25:35.305231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:35.330159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e3d62cbc-e6b6-4de8-a1c4-6ff70807251f tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1512 microversion: 2.1 time: 0.268382 Sep 18 06:25:35.331094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 89/171] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:35 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1512 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:36.046478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:36.048029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:36.055467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:36.056167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:36.057077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:36.058505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:36.058998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:36.059450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:36.228546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:36.241451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dcfb8d4b-74c0-4c76-8c04-4bb5cd6ec823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.207298 Sep 18 06:25:36.245568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 83/172] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:36 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:36.359755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:36.360871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:36.371872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:36.374276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:36.377002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:36.377905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:36.379979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:36.380624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:36.432107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:36.432562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:36.432995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:36.433904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lazy-loading 'fault' on Instance uuid 1d1894f2-f7ef-434c-8f30-0f99e862fd23 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:25:36.493191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93d23721dfd34020b659707b65eb2c33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:36.504523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fc8f0445-4e94-4ada-a602-fe33e6aa1331 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 200 len: 1351 microversion: 2.1 time: 0.149621 Sep 18 06:25:36.514729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 90/173] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:36 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 1351 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:37.265750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:37.267012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:37.272989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:37.273469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:37.274066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:37.275054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:37.275879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:37.276456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:37.487066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:37.501046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dcd454be-3784-42bf-ae54-f69312fae960 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.239332 Sep 18 06:25:37.517375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 84/174] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:37 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:37.534762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:37.536793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:37.547165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:37.547768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:37.548370 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:37.549423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:37.550034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:37.550779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:37.575404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] HTTP exception thrown: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Request method failure captured: Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 HTTP/1.1 Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmokytNwGgng-r63oySns4LusvJKQnU-WGZKreFv81euBwNkuPvTjglzR85xdU751NXxScEr7lHH4aKQMB-f6xbmZ-8-QALpUALDfN0sSeE0_cgjGpLayJhrujazUp3eMjqTmCI1xMSx2RSis-R-eUIOjGlNeTaalcBB0J8SW6ppVk Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:25:37.582443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-DeleteServersTestJSON-1865700061-project Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 93d23721dfd34020b659707b65eb2c33 Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-DeleteServersTestJSON-1865700061-project Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.583983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '1d1894f2-f7ef-434c-8f30-0f99e862fd23'} Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:25:37.584826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.586250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.589528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Returning 404 to user: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:25:37.591909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-03ee8ac7-8778-49a9-8f5d-d422f95f8678 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 404 len: 111 microversion: 2.1 time: 0.062186 Sep 18 06:25:37.592899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 91/175] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:37 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:25:37.733795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:37.736576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:37.743431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:37.744686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:37.745271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:37.746149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:37.746768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:37.747353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:37.762593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] HTTP exception thrown: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Request method failure captured: Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23/os-volume_attachments HTTP/1.1 Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmokytNwGgng-r63oySns4LusvJKQnU-WGZKreFv81euBwNkuPvTjglzR85xdU751NXxScEr7lHH4aKQMB-f6xbmZ-8-QALpUALDfN0sSeE0_cgjGpLayJhrujazUp3eMjqTmCI1xMSx2RSis-R-eUIOjGlNeTaalcBB0J8SW6ppVk Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:25:37.767015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-DeleteServersTestJSON-1865700061-project Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 93d23721dfd34020b659707b65eb2c33 Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-DeleteServersTestJSON-1865700061-project Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.769092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': '1d1894f2-f7ef-434c-8f30-0f99e862fd23'} Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:25:37.770923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:37.773222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:37.777262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Returning 404 to user: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:25:37.777262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5d4f7ace-ae9e-42e6-8b0b-8eb73e5a0ff9 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.039529 Sep 18 06:25:37.777262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 85/176] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:25:37 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23/os-volume_attachments => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:25:38.525901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:38.526519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:38.546614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:38.547196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:38.547576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:38.548322 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:38.548778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:38.549245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:38.788983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:38.798221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-de78d633-fe8c-49fd-afae-2c46ea2d7285 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.276583 Sep 18 06:25:38.799569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 92/177] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:38 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:39.154788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:39.155331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:39.169479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:39.169981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:39.170420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:39.179397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:39.180126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:39.180713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:39.224571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] HTTP exception thrown: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Request method failure captured: Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 HTTP/1.1 Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmokytNwGgng-r63oySns4LusvJKQnU-WGZKreFv81euBwNkuPvTjglzR85xdU751NXxScEr7lHH4aKQMB-f6xbmZ-8-QALpUALDfN0sSeE0_cgjGpLayJhrujazUp3eMjqTmCI1xMSx2RSis-R-eUIOjGlNeTaalcBB0J8SW6ppVk Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:25:39.229889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-DeleteServersTestJSON-1865700061-project Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 93d23721dfd34020b659707b65eb2c33 Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-DeleteServersTestJSON-1865700061-project Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.231161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '1d1894f2-f7ef-434c-8f30-0f99e862fd23'} Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:25:39.231848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.233005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.239655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Returning 404 to user: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:25:39.239655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eb8f99ab-72e8-4e48-bf77-c1fb03f05c0d tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 404 len: 111 microversion: 2.1 time: 0.083747 Sep 18 06:25:39.239655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 86/178] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:25:39 2022] DELETE /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:25:39.243607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:39.244278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] [instance: 1d1894f2-f7ef-434c-8f30-0f99e862fd23] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:39.255802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:39.256206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:39.256846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:39.258202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:39.259789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:39.276552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:39.293470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] HTTP exception thrown: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Request method failure captured: Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 HTTP/1.1 Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmokytNwGgng-r63oySns4LusvJKQnU-WGZKreFv81euBwNkuPvTjglzR85xdU751NXxScEr7lHH4aKQMB-f6xbmZ-8-QALpUALDfN0sSeE0_cgjGpLayJhrujazUp3eMjqTmCI1xMSx2RSis-R-eUIOjGlNeTaalcBB0J8SW6ppVk Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:25:39.295169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/22ba57c2ca3e4ff9a4b77d8cecbfa4e9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-DeleteServersTestJSON-1865700061 Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-DeleteServersTestJSON-1865700061-project Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 93d23721dfd34020b659707b65eb2c33 Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-DeleteServersTestJSON-1865700061-project Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.296378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '1d1894f2-f7ef-434c-8f30-0f99e862fd23'} Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:25:39.297072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. Sep 18 06:25:39.298131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:25:39.301121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] Returning 404 to user: Instance 1d1894f2-f7ef-434c-8f30-0f99e862fd23 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:25:39.302226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7f79a6b6-7e3f-4669-91be-89db3aeddea0 tempest-DeleteServersTestJSON-1865700061 tempest-DeleteServersTestJSON-1865700061-project] 10.222.0.20 "GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23" status: 404 len: 111 microversion: 2.1 time: 0.062904 Sep 18 06:25:39.303340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 93/179] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:39 2022] GET /compute/v2.1/servers/1d1894f2-f7ef-434c-8f30-0f99e862fd23 => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:25:39.823893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:39.824551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:39.832287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:39.834451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:39.835141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:39.836263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:39.836974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:39.838118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:39.872337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:40.210508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:40.235601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-30ca4528-d631-4900-a2f0-fdc03c20be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.415982 Sep 18 06:25:40.238750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 87/180] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:39 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:41.285433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:41.286178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:41.323167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:41.323716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.025s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:41.324166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:41.324913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:41.325374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:41.325804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:42.033308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:42.045772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-abceae56-e15f-4e0a-a1e2-42e3308c64cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.787863 Sep 18 06:25:42.050243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 94/181] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:41 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:43.078226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:43.079646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:43.087128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:43.088569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:43.089186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:43.090128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:43.090884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:43.091563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:43.426137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:43.439724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-36db5d32-db16-4d8f-ae36-f28378701811 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.365376 Sep 18 06:25:43.441412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 88/182] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:43 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:44.475705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:44.476387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:44.491618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:44.492122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:44.492750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:44.496584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:44.497084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:44.497525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:44.926806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:44.936696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cf9f2948-18d3-4f05-a7ec-fb5c02920d2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.465340 Sep 18 06:25:44.937354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 95/183] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:44 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:45.961255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:45.961865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:45.965025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:45.972269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:45.973400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:45.974000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:45.975049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:45.975706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:45.976168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:46.324220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:46.334835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4b581d43-edf1-4aa1-8316-2cc1120776c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.377404 Sep 18 06:25:46.343538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 89/184] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:45 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:47.366182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:47.366816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:47.377980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:47.378704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:47.379144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:47.379847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:47.380567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:47.380697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:47.644406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:47.656038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b226cfee-8c28-4492-8a8c-3802e816024a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.293401 Sep 18 06:25:47.659059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 96/185] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:47 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:47.807616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1578377682", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452"}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:47.900081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] validate_networks() for None {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:25:48.695865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:48.697213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:48.713873 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:48.724866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:48.726745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:48.730618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:48.731414 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:48.732840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:48.989582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:25:48.990371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:25:48.991046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:25:48.991460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:25:48.992729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:25:49.009656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:49.013035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:49.013813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:49.014837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:49.018114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:49.026285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:49.074557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Getting quotas for project 89b44de54291465a8b30343a11d72f78. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:25:49.090191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Getting quotas for user 4d1f91620b8e4d87b7ce4a015b0d447d and project 89b44de54291465a8b30343a11d72f78. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:25:49.123280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:25:49.126765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:25:49.145335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:49.160299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2b89bf0e-764a-4c85-8d53-bb12b5437625 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.468860 Sep 18 06:25:49.161214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 97/186] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:48 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:49.214164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fb145991-79c2-496e-9390-2e27845b73ac tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.411167 Sep 18 06:25:49.219340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 90/187] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:25:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 1417 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:25:49.230877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c86b2c0e-1aa9-4475-b133-3a79b484bd07 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:49.231922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c86b2c0e-1aa9-4475-b133-3a79b484bd07 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:49.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c86b2c0e-1aa9-4475-b133-3a79b484bd07 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:49.370719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c86b2c0e-1aa9-4475-b133-3a79b484bd07 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1215 microversion: 2.1 time: 0.143859 Sep 18 06:25:49.371306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 98/188] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:49 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1215 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:50.189659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:50.190447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:50.206137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:50.206671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:50.207119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:50.209998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:50.210500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:50.210966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:50.397676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:50.399155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:50.407496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:50.421329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:50.421787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:50.422530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:50.423116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:50.423431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:50.671425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:50.679761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c68ff4f2-de6a-4799-911a-b0094e9857c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.497658 Sep 18 06:25:50.681580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 91/189] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:50 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool [-] Exception during reset or similar: AssertionError: do not call blocking functions from the mainloop Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool Traceback (most recent call last): Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 739, in _finalize_fairy Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool fairy._reset(pool) Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 988, in _reset Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool pool._dialect.do_rollback(self) Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 682, in do_rollback Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool dbapi_connection.rollback() Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 480, in rollback Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool self._read_ok_packet() Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 443, in _read_ok_packet Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool pkt = self._read_packet() Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 692, in _read_packet Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool packet_header = self._read_bytes(4) Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 732, in _read_bytes Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool data = self._rfile.read(num_bytes) Sep 18 06:25:50.832314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/lib/python3.8/socket.py", line 669, in readinto Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool return self._sock.recv_into(b) Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/eventlet/greenio/base.py", line 376, in recv_into Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool return self._recv_loop(self.fd.recv_into, 0, buffer, nbytes, flags) Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/eventlet/greenio/base.py", line 364, in _recv_loop Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool self._read_trampoline() Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/eventlet/greenio/base.py", line 332, in _read_trampoline Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool self._trampoline( Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/eventlet/greenio/base.py", line 211, in _trampoline Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool return trampoline(fd, read=read, write=write, timeout=timeout, Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/__init__.py", line 141, in trampoline Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool assert hub.greenlet is not current, 'do not call blocking functions from the mainloop' Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool AssertionError: do not call blocking functions from the mainloop Sep 18 06:25:50.833551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR sqlalchemy.pool.impl.QueuePool Sep 18 06:25:50.834809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Exception ignored in: . at 0x7fe67090bee0> Sep 18 06:25:50.834969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Traceback (most recent call last): Sep 18 06:25:50.835108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 488, in Sep 18 06:25:50.835566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: and _finalize_fairy(None, rec, pool, ref, echo, True), Sep 18 06:25:50.835713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 774, in _finalize_fairy Sep 18 06:25:50.836217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: connection_record.checkin() Sep 18 06:25:50.836413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 521, in checkin Sep 18 06:25:50.836821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: pool.dispatch.checkin(connection, self) Sep 18 06:25:50.836998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/event/attr.py", line 343, in __call__ Sep 18 06:25:50.840119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: fn(*args, **kw) Sep 18 06:25:50.840738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py", line 51, in _thread_yield Sep 18 06:25:50.854164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: time.sleep(0) Sep 18 06:25:50.854164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: File "/usr/local/lib/python3.8/dist-packages/eventlet/greenthread.py", line 33, in sleep Sep 18 06:25:50.860601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: assert hub.greenlet is not current, 'do not call blocking functions from the mainloop' Sep 18 06:25:50.860864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: AssertionError: do not call blocking functions from the mainloop Sep 18 06:25:51.104202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:51.129821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-45298928-e760-49c4-aa78-de8018beb833 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1267 microversion: 2.1 time: 0.727810 Sep 18 06:25:51.131908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 99/190] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:50 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1267 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:51.707566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:51.708407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:51.719567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:51.721511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:51.722320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:51.724507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:51.725959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:51.728988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:52.008475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:52.020413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2673972e-722f-47c5-8a1d-929a467a8e4c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.316540 Sep 18 06:25:52.036697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 92/191] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:51 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:52.156612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:52.157278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:52.171901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:52.172576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:52.173413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:52.174127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:52.174590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:52.186718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:52.306345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:52.322120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4a2e598f-bf39-4904-885e-ccfc091f3acd tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1275 microversion: 2.1 time: 0.169461 Sep 18 06:25:52.326539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 100/192] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:52 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1275 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:53.055852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:53.060091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:53.078123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:53.078872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:53.081759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:53.086365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:53.087758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:53.089232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:53.351801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:53.359856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:53.360968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:53.366294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-688f3955-47d9-4eb5-ac2f-3c5887fe3125 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.314781 Sep 18 06:25:53.379555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:53.380472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:53.383258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 93/193] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:53 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:53.407255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.026s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:53.408053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:53.408488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:53.408911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:53.451397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:53.620310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:53.640545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cf3b06f0-2233-465d-ae79-2897acaa87c6 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1275 microversion: 2.1 time: 0.289784 Sep 18 06:25:53.641458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 101/194] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:53 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1275 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:54.409284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:54.409917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:54.430239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:54.447712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:54.448275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:54.449046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:54.449524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:54.449956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:54.664602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:54.665959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:54.675757 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:54.680872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:54.686343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:54.692995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:54.693708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:54.696828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:54.857418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:54.868440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7b35872e-06c1-4baa-8b57-e0c051dd1421 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1285 microversion: 2.1 time: 0.208291 Sep 18 06:25:54.873657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 102/195] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:54 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1285 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:54.997726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:55.017095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dfda3330-7ce1-4e16-abaa-452ad1624bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.614239 Sep 18 06:25:55.020073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 94/196] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:54 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:55.901665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:55.903002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:55.915005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:55.915872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:55.916519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:55.917436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:55.919231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:55.920346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:56.042531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:56.043705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:56.063330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:56.063966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:56.064402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:56.065308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:56.067782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:56.068593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:56.126112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:56.136085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9a5762e4-ba95-4f3c-bc11-dc685c238399 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1273 microversion: 2.1 time: 0.239599 Sep 18 06:25:56.140446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 103/197] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:55 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1273 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:56.545033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:56.573660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d14abcc4-52eb-4afd-829c-bef3327fe8d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.535928 Sep 18 06:25:56.576999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 95/198] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:56 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:25:57.169374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:57.170818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:57.193386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:57.194079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:57.195723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:57.197528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:57.200483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:57.201150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:57.557896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:57.566112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f64115ab-1d43-4552-a038-87b8d3ef84f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1315 microversion: 2.1 time: 0.401690 Sep 18 06:25:57.569648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 104/199] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:57 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1315 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:57.613972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:57.615190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:57.622654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:57.623497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:57.624170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:57.625362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:57.626030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:57.626900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:57.892495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:57.903982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3b660fd0-e911-4c62-b1dd-73eded185975 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1521 microversion: 2.1 time: 0.305550 Sep 18 06:25:57.905291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 96/200] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:57 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1521 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:58.220330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-31fa9ce0-2c7e-4a4f-ad3b-6efd965a9757 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'create', calling method: .version_select at 0x7fe670680af0>, body: {"keypair": {"name": "tempest-keypair-1422298950", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCAiit8ncmM/atSNPNeOv5jZdKYNFGmOCu+a0Cgwc+sJKxZdfivr2rGcIpkQPymAXXjl6Ph5I+DqTLpiZ8TXzy26rFekJn2AnLtEqOTUBVpCNv4miY42fMnRQtI9NOPTPA=="}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:58.279918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-31fa9ce0-2c7e-4a4f-ad3b-6efd965a9757 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Getting quotas for project 6fbf7eeb158e4a72bd8133faa1366e0d. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:25:58.292566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-31fa9ce0-2c7e-4a4f-ad3b-6efd965a9757 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Getting quotas for user 00399347c76d451d94d45f4edc3cf416 and project 6fbf7eeb158e4a72bd8133faa1366e0d. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:25:58.358864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-31fa9ce0-2c7e-4a4f-ad3b-6efd965a9757 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.511326 Sep 18 06:25:58.361636 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 105/201] 10.222.0.20 () {62 vars in 1200 bytes} [Sun Sep 18 06:25:57 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 515 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Sep 18 06:25:58.609412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:58.610615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:58.630210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:58.631683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:58.634062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:58.635795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:58.638313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:58.639535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:58.931496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:58.932495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:58.952677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:58.954450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:58.956433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:58.957601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:58.959676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:58.961556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:59.048475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:59.114733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=6 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:59.169636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:59.179556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c16d63c6-51c1-4dfe-a691-54018222d877 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1315 microversion: 2.1 time: 0.579030 Sep 18 06:25:59.180526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 97/202] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:58 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1315 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:59.235108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-6ebefe9c-b063-45b3-9d56-e83e67d9f159 req-8cb82ec6-30cf-4d69-a623-43a35197a714 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ab927d7-e82e-4cd4-a68d-a0895accce99", "tag": "35fe7697-bcc5-4b28-ab17-e6ce8f0e3fd9"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:25:59.254231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6ebefe9c-b063-45b3-9d56-e83e67d9f159 req-8cb82ec6-30cf-4d69-a623-43a35197a714 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:59.254898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6ebefe9c-b063-45b3-9d56-e83e67d9f159 req-8cb82ec6-30cf-4d69-a623-43a35197a714 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:59.255489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6ebefe9c-b063-45b3-9d56-e83e67d9f159 req-8cb82ec6-30cf-4d69-a623-43a35197a714 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:59.281626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-6ebefe9c-b063-45b3-9d56-e83e67d9f159 req-8cb82ec6-30cf-4d69-a623-43a35197a714 service nova] Creating event network-changed:35fe7697-bcc5-4b28-ab17-e6ce8f0e3fd9 for instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 on hv-cinder-85535 Sep 18 06:25:59.292234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-6ebefe9c-b063-45b3-9d56-e83e67d9f159 req-8cb82ec6-30cf-4d69-a623-43a35197a714 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061271 Sep 18 06:25:59.292830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 98/203] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:25:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:25:59.536827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:25:59.547129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-caf7e3c1-f6ee-4a44-a59c-eb302a614656 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1501 microversion: 2.1 time: 0.621157 Sep 18 06:25:59.559257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 106/204] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:58 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1501 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:25:59.574487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:25:59.579241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:25:59.590392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:59.597412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:59.598292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:25:59.599269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:25:59.599910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:25:59.600530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:00.115833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:00.134243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e282582f-fd94-4298-952e-107555a517e9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1516 microversion: 2.1 time: 0.563522 Sep 18 06:26:00.147570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 99/205] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:25:59 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1516 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:00.209573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:00.210196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:00.222658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:00.224960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:00.226787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:00.227998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:00.229671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:00.230309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:00.500437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:00.511900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9df06001-9527-4f13-b949-951efbe6467a tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1315 microversion: 2.1 time: 0.310107 Sep 18 06:26:00.513449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 107/206] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:00 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1315 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:01.544351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:01.545903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:01.557219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:01.558413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:01.559135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:01.561496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:01.562304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:01.563812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:01.832149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:01.838813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:01.844549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:01.849990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4544624e-c463-4166-9a58-46cd6c54921d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1315 microversion: 2.1 time: 0.311691 Sep 18 06:26:01.864241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 100/207] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:01 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1315 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:01.868765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:01.872066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:01.872784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:01.874710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:01.877278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:01.879745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:01.880671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:01.946490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lazy-loading 'flavor' on Instance uuid 1dd95cf5-d6be-4de9-9941-8b1d35855bcb {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:26:02.184482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-aeed60bb-a427-4b71-bd8d-641197d56e77 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb/action" status: 200 len: 32 microversion: 2.1 time: 0.355732 Sep 18 06:26:02.189441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 108/208] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:26:01 2022] POST /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb/action => generated 32 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 06:26:02.207636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:02.212993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:02.234988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:02.249568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:02.251179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:02.251941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:02.252440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:02.252855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:02.868702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1904031791", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b0343989-7ee1-40ca-87bc-aae149bc4efe"}], "security_groups": [{"name": "tempest-securitygroup--721243334"}], "key_name": "tempest-keypair-1422298950", "user_data": "}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:03.059737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:03.068056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3f4f0a6e-cbae-45a1-a292-6a7f0d08449d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.876523 Sep 18 06:26:03.081664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 101/209] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:02 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:03.086799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:03.088268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:03.111678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:03.112093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:03.112563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:03.113281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:03.113732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:03.116214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:03.345209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] validate_networks() for [('b0343989-7ee1-40ca-87bc-aae149bc4efe', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:26:03.692985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:03.712821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c31e7719-3182-434f-a859-1bc1c06fd258 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1315 microversion: 2.1 time: 0.628167 Sep 18 06:26:03.716026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 102/210] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:03 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1315 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:04.094718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:04.096365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:04.109268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:04.109760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:04.110201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:04.113310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:04.113720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:04.116695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:04.124844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:26:04.125923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:26:04.126599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:26:04.127258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:26:04.128069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:26:04.149113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:04.151435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:04.152138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:04.153027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:04.155087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:04.157669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:04.200351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Getting quotas for project 6fbf7eeb158e4a72bd8133faa1366e0d. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:04.210267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Getting quotas for user 00399347c76d451d94d45f4edc3cf416 and project 6fbf7eeb158e4a72bd8133faa1366e0d. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:04.226101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:26:04.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:26:04.301716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-76af5615-a669-4003-ade1-2bf49b4ddb2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 1.439196 Sep 18 06:26:04.304445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 109/211] 10.222.0.20 () {62 vars in 1188 bytes} [Sun Sep 18 06:26:02 2022] POST /compute/v2.1/servers => generated 401 bytes in 1443 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 18 06:26:04.325621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c003e25a-49ec-47bb-87b2-365a35a5440a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:04.326245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c003e25a-49ec-47bb-87b2-365a35a5440a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:04.494835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c003e25a-49ec-47bb-87b2-365a35a5440a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:04.503566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:04.504373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c003e25a-49ec-47bb-87b2-365a35a5440a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:04.505843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c003e25a-49ec-47bb-87b2-365a35a5440a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:04.507022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c003e25a-49ec-47bb-87b2-365a35a5440a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1282 microversion: 2.20 time: 0.194840 Sep 18 06:26:04.516127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6d1867d8-97d5-437f-bc45-d18a99b5026b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.425111 Sep 18 06:26:04.517816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 103/212] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:04 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:04.525522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 110/213] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:04 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1282 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:26:04.737181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:04.737801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:04.744836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:04.745333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:04.745774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:04.746475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:04.746960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:04.747386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:04.996455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:05.015075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-830ff6d2-4a23-48eb-a1dc-57252c2e2a96 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1315 microversion: 2.1 time: 0.282068 Sep 18 06:26:05.016715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 104/214] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:04 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1315 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:05.097436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-202648cd-e0ef-4890-a961-4e8ef708fa04 req-0b727f92-e490-49a6-ae91-0bcff3a566c8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ab927d7-e82e-4cd4-a68d-a0895accce99", "name": "network-vif-plugged", "status": "completed", "tag": "35fe7697-bcc5-4b28-ab17-e6ce8f0e3fd9"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:05.166971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-202648cd-e0ef-4890-a961-4e8ef708fa04 req-0b727f92-e490-49a6-ae91-0bcff3a566c8 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:05.173551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-202648cd-e0ef-4890-a961-4e8ef708fa04 req-0b727f92-e490-49a6-ae91-0bcff3a566c8 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:05.175517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-202648cd-e0ef-4890-a961-4e8ef708fa04 req-0b727f92-e490-49a6-ae91-0bcff3a566c8 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:05.250312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-202648cd-e0ef-4890-a961-4e8ef708fa04 req-0b727f92-e490-49a6-ae91-0bcff3a566c8 service nova] Creating event network-vif-plugged:35fe7697-bcc5-4b28-ab17-e6ce8f0e3fd9 for instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 on hv-cinder-85535 Sep 18 06:26:05.372995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-202648cd-e0ef-4890-a961-4e8ef708fa04 req-0b727f92-e490-49a6-ae91-0bcff3a566c8 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.279468 Sep 18 06:26:05.373758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 111/215] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:05.542383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:05.543762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:05.553827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:05.554599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:05.557686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:05.560271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:05.561155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:05.564508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:05.566411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:05.567083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:05.568548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:05.569047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:05.569481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:05.570180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:05.570732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:05.577707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:05.590982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:05.753924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:05.766810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:05.771912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:05.776206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cfd807e5-675f-4a2b-92e4-dc1775a0bcbe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1286 microversion: 2.20 time: 0.226777 Sep 18 06:26:05.777066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 112/216] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:05 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1286 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:26:05.931169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:05.972324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-83d62158-371d-443c-b3b8-08ca57aecdb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.435224 Sep 18 06:26:05.974106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 105/217] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:05 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:06.043886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:06.044856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:06.070063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:06.071005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:06.071739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:06.072653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:06.073254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:06.073832 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:06.388425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:06.399997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bf5de7e4-8ed7-4581-9bed-469247d6fced tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Action: 'create', calling method: .version_select at 0x7fe670671790>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1546923934", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLllmgvEvOGVqjQr1B9+B9AqGliUuynpmasHAFaghZBPw0SaqdPfwKZtT+eeAM3kP1d7Poz/nrzOMtuOlXSjC/PX6AN4kRwZyVTVsNHzIgkBKwrecapY9f8UAEKqkWjO2g=="}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:06.405234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-409e1d93-aa53-4764-a1ed-077268d51017 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1315 microversion: 2.1 time: 0.367773 Sep 18 06:26:06.406023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 113/218] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:06 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1315 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:06.422420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-bf5de7e4-8ed7-4581-9bed-469247d6fced tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Getting quotas for project 2af8940da97b4298a92e1d45f70cfe29. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:06.426541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-bf5de7e4-8ed7-4581-9bed-469247d6fced tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Getting quotas for user fd862cc69267470580ef23e0223742e0 and project 2af8940da97b4298a92e1d45f70cfe29. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:06.474664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bf5de7e4-8ed7-4581-9bed-469247d6fced tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.078551 Sep 18 06:26:06.475278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 106/219] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:26:06 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:06.588411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1371513873", "imageRef": "6ee8097f-ea74-4512-829f-e665733bb114", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1546923934", "networks": [{"uuid": "894c4893-5950-4d31-8c69-9cd5060deb9d"}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:06.648397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] validate_networks() for [('894c4893-5950-4d31-8c69-9cd5060deb9d', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:26:06.805265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:06.806087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:06.812691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:06.813176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:06.813606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:06.814296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:06.814790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:06.815236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:06.889351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:06.899244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:06.900769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:06.901788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6b5d9945-83e1-4233-b65b-3e64de03e29c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1334 microversion: 2.20 time: 0.100682 Sep 18 06:26:06.902338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 107/220] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:06 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1334 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:26:06.998018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:06.998975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:07.013395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.013395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.013395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.013395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.013738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.013738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.127821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:26:07.129524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:26:07.130473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:26:07.132546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:26:07.133290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:26:07.143215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.143703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.144132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.145128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.145735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.146304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.177934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Getting quotas for project 2af8940da97b4298a92e1d45f70cfe29. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:07.183684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Getting quotas for user fd862cc69267470580ef23e0223742e0 and project 2af8940da97b4298a92e1d45f70cfe29. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:07.200954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:26:07.203980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='6ee8097f-ea74-4512-829f-e665733bb114',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:26:07.207175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:07.216131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f98b684f-2011-4c57-a125-7f4738f90bad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.223463 Sep 18 06:26:07.217023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 108/221] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:06 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:07.266435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4158ac2a-2cab-461d-bee8-df02d8c60ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.682812 Sep 18 06:26:07.267305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 114/222] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:26:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 684 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 06:26:07.280435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4f982f36-c931-4437-be45-a5de895499d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:07.281517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4f982f36-c931-4437-be45-a5de895499d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:07.374108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4f982f36-c931-4437-be45-a5de895499d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:07.381788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4f982f36-c931-4437-be45-a5de895499d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1259 microversion: 2.1 time: 0.106452 Sep 18 06:26:07.383659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 109/223] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:07 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1259 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:07.429120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:07.429744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:07.437859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.438878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.439181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.440069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.441034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.441604 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.731290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:07.740381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0ece19a5-2c32-412a-bb75-a0a2576525e2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1315 microversion: 2.1 time: 0.315357 Sep 18 06:26:07.740954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 115/224] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:07 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1315 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:07.855883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:07.856685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:07.866772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.867693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.868360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.869323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.870034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.870652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.906544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c80955f5-c892-4226-b461-d3e942746e5e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/ips" status: 200 len: 144 microversion: 2.1 time: 0.055315 Sep 18 06:26:07.907223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 110/225] 10.222.0.20 () {58 vars in 1248 bytes} [Sun Sep 18 06:26:07 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/ips => generated 144 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:07.925990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:07.926643 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:07.934550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.935374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.935989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:07.936694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:07.937150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:07.937574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:08.034814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:08.043986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:08.045603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:08.047090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5fb5956c-bb44-4d5c-9c73-1ed4cc83b866 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1340 microversion: 2.20 time: 0.124994 Sep 18 06:26:08.047722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 116/226] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:07 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1340 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:26:08.240770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:08.241703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:08.252722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:08.253414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:08.254260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:08.255658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:08.256333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:08.256913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:08.409289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:08.409907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:08.418020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:08.418970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:08.420192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:08.431920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:08.440098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:08.442982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:08.801925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:08.804365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:08.816800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-73688ab9-6b24-45c3-93e6-5bbfcb5503d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.581074 Sep 18 06:26:08.817544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 111/227] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:08 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:08.823614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4c4b7cd0-6d01-4b0f-a3bf-97eb727ebca6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1311 microversion: 2.1 time: 0.419116 Sep 18 06:26:08.825948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:08.826454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:08.826963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 117/228] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:08 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1311 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:08.848859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:08.849396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:08.849790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:08.850888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:08.851027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:08.852187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.072996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:09.073999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:09.081553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.082292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.082906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.083784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.085024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.085614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.126046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:09.138368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c7280bc1-cd47-45bf-b6b7-7b0e637a4c37 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1465 microversion: 2.1 time: 0.319573 Sep 18 06:26:09.139868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 112/229] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:08 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1465 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:09.164855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:09.181401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:09.183006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:09.184253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-db135667-0e18-4c21-b022-5440cd97cb47 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1340 microversion: 2.20 time: 0.115537 Sep 18 06:26:09.198898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 118/230] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:09 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1340 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:09.217654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:09.218318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:09.226207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.226822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.227981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.228877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.229340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.229816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.281150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:26:09.414343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-48bdc4a1-50c8-44f6-b323-7baa04bd8f72 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "DELETE /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 204 len: 0 microversion: 2.1 time: 0.202838 Sep 18 06:26:09.415573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 113/231] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:26:09 2022] DELETE /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:26:09.441427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:09.442814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:09.468850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.469733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.470332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.471292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.471906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.472474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.764776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:09.772248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:09.783579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f7f82bd8-8c50-4c83-a409-0de2a5c980d1 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1471 microversion: 2.1 time: 0.363142 Sep 18 06:26:09.788056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 119/232] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:09 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1471 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:09.852238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:09.853286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:09.866096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.866944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.868322 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.869224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.873352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.874022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.880221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:09.881309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:09.904741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.907733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.908299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:09.909190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:09.910525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:09.910998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:10.331195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:10.338161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:10.346341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-887d3c42-8945-4d8a-8256-8c7cf2bb0e1c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.496937 Sep 18 06:26:10.351975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8243b3bb-7a00-4219-8dc9-ca1f1d1b195f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.503427 Sep 18 06:26:10.352577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 120/233] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:09 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:10.364397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 114/234] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:09 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:10.365984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:10.366744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:10.373522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:10.374539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:10.375614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:10.376785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:10.377614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:10.378664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:10.705559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:10.717248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:10.719635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:10.720905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b9a6b9b5-9854-4b69-bba2-a221e9a5d8a0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.367132 Sep 18 06:26:10.721706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 121/235] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:10.813108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:10.813891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:10.821269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:10.821898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:10.822214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:10.822919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:10.823361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:10.823780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:11.002752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-72ec2943-19c2-4456-8165-929df6b67391 req-08fdc81d-4fec-42b1-aef0-2020818e5c5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5ed3cfa3-6093-4c26-9877-01470acd2e16", "tag": "84893f57-6d56-4075-9e20-5bd22c657e44"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:11.021616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-72ec2943-19c2-4456-8165-929df6b67391 req-08fdc81d-4fec-42b1-aef0-2020818e5c5e service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:11.022091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-72ec2943-19c2-4456-8165-929df6b67391 req-08fdc81d-4fec-42b1-aef0-2020818e5c5e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:11.022516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-72ec2943-19c2-4456-8165-929df6b67391 req-08fdc81d-4fec-42b1-aef0-2020818e5c5e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:11.049528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-72ec2943-19c2-4456-8165-929df6b67391 req-08fdc81d-4fec-42b1-aef0-2020818e5c5e service nova] Creating event network-changed:84893f57-6d56-4075-9e20-5bd22c657e44 for instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 on hv-cinder-85535 Sep 18 06:26:11.056423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-72ec2943-19c2-4456-8165-929df6b67391 req-08fdc81d-4fec-42b1-aef0-2020818e5c5e service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057767 Sep 18 06:26:11.056953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 122/236] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:11.152455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:11.161432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-83e57243-6974-472e-9962-7b22b8be707d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1471 microversion: 2.1 time: 0.353250 Sep 18 06:26:11.162698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 115/237] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:10 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1471 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:11.373435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:11.374592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:11.375677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:11.376563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:11.384541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:11.384739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:11.385123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:11.385550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:11.386079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:11.386524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:11.386974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:11.387268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:11.387682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:11.388654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:11.389465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:11.390420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:11.657756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:11.670156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eb721bb0-1fba-42b0-b008-d976ec56b5ec tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.298450 Sep 18 06:26:11.671559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 116/238] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:11.677797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:11.687892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-07d07a69-4168-4046-92e0-f534a6362115 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.318834 Sep 18 06:26:11.689055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 123/239] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:11 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:11.745717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:11.746559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:11.753189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:11.753759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:11.754197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:11.754991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:11.755368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:11.755795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:11.927572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:11.940854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:11.942662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:11.943788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-69ffb888-c429-4d01-9579-e82e2d32f86a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.202418 Sep 18 06:26:11.944346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 117/240] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:11 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:12.186099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:12.187462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:12.194010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:12.194730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:12.196197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:12.200203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:12.200763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:12.201232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:12.335543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:12.343361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8a0839d2-f3b1-471f-9aab-bc018efc45aa tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1471 microversion: 2.1 time: 0.161280 Sep 18 06:26:12.343924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 124/241] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:12 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1471 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:12.700998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:12.702471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:12.725933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:12.727699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:12.732559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:12.735381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:12.737210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:12.738341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:12.740122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:12.743044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:12.743695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:12.743962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:12.744443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:12.745178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:12.745627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:12.746218 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:13.007666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:13.022342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-03ae83f7-731c-4c75-aa2f-533942d2b84c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.326687 Sep 18 06:26:13.024354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 118/242] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:12 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:13.031907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:13.033036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:13.044163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:13.046136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:13.047260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:13.048405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:13.048968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:13.049690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:13.054134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:13.071205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5211a9c9-e343-4251-8934-6bb0580a6e92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.348010 Sep 18 06:26:13.074337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 125/243] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:12 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:13.262436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:13.279435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:13.280886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:13.282114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-986f0011-2f4d-4d27-936a-3d4c64cdbc20 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.255561 Sep 18 06:26:13.282570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 119/244] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:13 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:13.366373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:13.367190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:13.374289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:13.385305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:13.385830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:13.386787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:13.387346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:13.388455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:13.570987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:13.579030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c67d3400-c31f-463c-80b8-0b5f3dfed057 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1471 microversion: 2.1 time: 0.216573 Sep 18 06:26:13.579613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 126/245] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:13 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1471 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:14.045787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:14.046362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:14.052216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:14.052913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:14.053343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:14.054039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:14.054524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:14.055485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:14.097007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:14.097897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:14.105997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:14.106632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:14.106978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:14.107696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:14.108151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:14.108603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:14.241491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:14.251197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-879b2bd9-d4bf-44a5-9bb9-c164f829d1d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.208584 Sep 18 06:26:14.252004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 120/246] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:14.302978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:14.308464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-95ec4f32-4512-443c-a754-58e8ab2aae48 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.215279 Sep 18 06:26:14.311465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:14.312478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:14.320224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:14.321108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:14.322534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:14.323582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:14.324320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:14.326019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 127/247] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:14 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:14.327194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:14.479528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:14.487924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:14.489454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:14.490665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b948f8c8-338e-4390-86e2-90d3a00d7239 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.183511 Sep 18 06:26:14.502605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 121/248] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:14 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:14.603629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:14.604429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:14.611004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:14.611577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:14.612078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:14.612784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:14.613265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:14.613690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:14.750116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:14.759333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-01af51bb-4e11-47a3-9384-8e7f290f5620 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1471 microversion: 2.1 time: 0.160523 Sep 18 06:26:14.763013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 128/249] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:14 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1471 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:15.273432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:15.274120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:15.280185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:15.280666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:15.281094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:15.281961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:15.282536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:15.282984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:15.302084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:15.338097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:15.339254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:15.347402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:15.348029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:15.348572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:15.349384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:15.349988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:15.350721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:15.478910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:15.487698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ef4fe0d4-04d2-4936-ab59-dbfda7b65a70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.217428 Sep 18 06:26:15.490848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 122/250] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:15 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:15.517243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:15.518596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:15.520502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:15.529509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:15.530492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:15.531189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:15.532660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:15.533423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:15.534044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:15.567121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:15.579851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7d3026f5-fb19-4723-a6b7-ad44f8127814 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.245756 Sep 18 06:26:15.581060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 129/251] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:15 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:15.707959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:15.717010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:15.718458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:15.719835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fc19c42e-93cd-43dc-9eaa-5beaf89eb62f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.206448 Sep 18 06:26:15.720386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 123/252] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:15 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:26:15.782927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:15.783482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:15.790448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:15.791208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:15.791788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:15.792725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:15.793172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:15.793753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:15.926506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:15.934288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-136be13f-15d9-4b4f-94f9-64e778f3bb65 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1471 microversion: 2.1 time: 0.155411 Sep 18 06:26:15.935114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 130/253] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:15 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1471 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:16.519228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:16.520413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:16.537087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:16.537800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:16.538228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:16.539734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:16.540333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:16.541055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:16.582692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b853ab37-b2a4-461c-92c4-5966e90df9ca"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:16.589032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:16.601511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:16.604249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:16.607914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:16.608674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:16.609310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:16.612175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:16.721599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:16.733318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-53426e23-af9d-42a7-9519-115ebef1565d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.217530 Sep 18 06:26:16.734890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 124/254] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:16.742239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:16.743073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:16.755737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:16.756431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:16.757252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:16.758322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:16.759025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:16.759894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:16.788171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-296adc15-5b69-433d-9e6e-dfdced8ca6a3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/ips" status: 200 len: 150 microversion: 2.1 time: 0.051989 Sep 18 06:26:16.789510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 125/255] 10.222.0.20 () {58 vars in 1248 bytes} [Sun Sep 18 06:26:16 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/ips => generated 150 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:16.794286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:16.795261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:16.802237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:16.803156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:16.803816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:16.805329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:16.806024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:16.806779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:17.004993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:17.026304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-40b6fa14-8d5b-4fc8-af4b-cc79fbb8e7af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.235483 Sep 18 06:26:17.027278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 126/256] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:16 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:17.065761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:17.066863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:17.083857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:17.083857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:17.084383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:17.085612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:17.086358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:17.087152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:17.155760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:17.197547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=20 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:17.378421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=6 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:17.457906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:17.468710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:17.473620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:17.473620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c3ad00fa-51cf-41da-bfd0-19608fa5d1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.428982 Sep 18 06:26:17.482747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 127/257] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:17 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:17.494075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:17.494938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:17.507657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:17.508225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:17.508721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:17.509588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:17.510003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:17.510434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:17.690747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:17.700496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4df8300c-0a27-4750-8bf2-41ca2294ee9e tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1471 microversion: 2.1 time: 0.214051 Sep 18 06:26:17.701311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 128/258] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:17 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1471 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:17.759917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:17.760540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:17.767105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:17.767720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:17.768286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:17.768995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:17.769514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:17.770279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:17.925630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:17.933431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4811025a-467b-45d5-8e22-9b9a2f66b2b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.177845 Sep 18 06:26:17.933995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 129/259] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:17 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:18.068655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:18.069262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:18.076222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:18.076704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:18.077138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:18.077826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:18.078277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:18.078729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:18.227597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:18.235486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2b6d261e-d406-4f4d-b9a9-bcab431ba2cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.170659 Sep 18 06:26:18.236072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 130/260] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:18 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:18.497850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:18.498453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:18.504653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:18.505124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:18.505555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:18.507080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:18.507544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:18.507997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:18.650567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:18.659552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:18.661013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:18.662128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-208e3604-64df-4373-a62e-1b5d28644e9f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.168260 Sep 18 06:26:18.668622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 131/261] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:18 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:26:18.682854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2e3b9bc4-9d2a-401b-a62a-91136b41b8f4"}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:18.686019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:18.692694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:18.693852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:18.694457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:18.695222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:18.695681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:18.696104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:21.690921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a498950a-882c-4b05-8f28-1b2d84a8599f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 5.111053 Sep 18 06:26:21.693007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 131/262] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:26:16 2022] POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/os-volume_attachments => generated 194 bytes in 5115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:21.714108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:21.714866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:21.732786 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:21.733494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:21.735623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:21.747579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:21.748084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:21.748576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:21.767761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e03378a3-5847-4de6-ae7b-40f7032684c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.096719 Sep 18 06:26:21.772457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 132/263] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:26:18 2022] POST /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 194 bytes in 3103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:21.784803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:21.786817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:21.801940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:21.803707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:21.804989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:21.806060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:21.806545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:21.809120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:22.450175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:22.461180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3001a90e-c25d-40e5-b6bd-5316bf8394b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.685772 Sep 18 06:26:22.462631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 133/264] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:21 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:22.470490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:22.471637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:22.484861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:22.487576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:22.489462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:22.490094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:22.493656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:22.494295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:22.494947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:22.497997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6e24785a-aac7-4073-b291-6da0629780a2 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1471 microversion: 2.1 time: 0.791416 Sep 18 06:26:22.499623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 132/265] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:21 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1471 bytes in 806 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:22.518334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:22.520195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:22.531728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:22.536130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:22.538032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:22.540065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:22.541119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:22.542037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:22.806221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:22.814228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-620aa2d5-cbd2-4b7a-93a7-5c10fa2063d3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.350276 Sep 18 06:26:22.814901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 134/266] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:22 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:22.819344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-2286f5a9-f319-410f-be16-fb7a656c7526 req-11e8d148-b3db-4380-81bd-32153cca8636 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b4bf8fc2-b57b-4198-939c-bd88739c785e", "tag": "1956a12e-1f36-454a-baa5-f949db978b94"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:22.829066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-2286f5a9-f319-410f-be16-fb7a656c7526 req-11e8d148-b3db-4380-81bd-32153cca8636 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:22.829747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-2286f5a9-f319-410f-be16-fb7a656c7526 req-11e8d148-b3db-4380-81bd-32153cca8636 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:22.830170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-2286f5a9-f319-410f-be16-fb7a656c7526 req-11e8d148-b3db-4380-81bd-32153cca8636 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:22.842134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-2286f5a9-f319-410f-be16-fb7a656c7526 req-11e8d148-b3db-4380-81bd-32153cca8636 service nova] Creating event network-changed:1956a12e-1f36-454a-baa5-f949db978b94 for instance b4bf8fc2-b57b-4198-939c-bd88739c785e on hv-cinder-85535 Sep 18 06:26:22.846961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-2286f5a9-f319-410f-be16-fb7a656c7526 req-11e8d148-b3db-4380-81bd-32153cca8636 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030918 Sep 18 06:26:22.847562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 135/267] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:22.936903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:22.950990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-36fed777-f78c-45a3-8d1a-93eae68c260e req-e45bf0de-b4d5-4e83-aa46-c010ddc1f22b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ab927d7-e82e-4cd4-a68d-a0895accce99", "name": "network-vif-deleted", "tag": "35fe7697-bcc5-4b28-ab17-e6ce8f0e3fd9"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:22.956751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:22.958764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:22.960034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-85562234-81d7-47e6-934e-038dba4430cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.459258 Sep 18 06:26:22.960588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 133/268] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:22 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:22.961321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-36fed777-f78c-45a3-8d1a-93eae68c260e req-e45bf0de-b4d5-4e83-aa46-c010ddc1f22b service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:22.961756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-36fed777-f78c-45a3-8d1a-93eae68c260e req-e45bf0de-b4d5-4e83-aa46-c010ddc1f22b service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:22.962194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-36fed777-f78c-45a3-8d1a-93eae68c260e req-e45bf0de-b4d5-4e83-aa46-c010ddc1f22b service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:22.973760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-36fed777-f78c-45a3-8d1a-93eae68c260e req-e45bf0de-b4d5-4e83-aa46-c010ddc1f22b service nova] Creating event network-vif-deleted:35fe7697-bcc5-4b28-ab17-e6ce8f0e3fd9 for instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 on hv-cinder-85535 Sep 18 06:26:22.979508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-36fed777-f78c-45a3-8d1a-93eae68c260e req-e45bf0de-b4d5-4e83-aa46-c010ddc1f22b service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032583 Sep 18 06:26:22.980110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 136/269] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:23.486501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:23.487167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:23.493989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:23.494476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:23.494936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:23.495618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:23.496068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:23.496490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:23.522029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:23.522783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:23.529690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:23.530112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:23.530547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:23.531271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:23.531748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:23.532176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:23.608908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:23.617191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-09b1d333-9ff1-4878-8365-00d9f3edfc9f tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1429 microversion: 2.1 time: 0.099027 Sep 18 06:26:23.617763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 137/270] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:23 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1429 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:23.645620 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:23.653265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1d8ce172-0ccc-42c2-a62c-a228df1b6944 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.170764 Sep 18 06:26:23.653821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 134/271] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:23 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:23.837808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:23.838332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:23.844472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:23.844948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:23.845382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:23.846062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:23.846525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:23.846952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:23.972200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:23.981837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7a7ade36-c466-4b20-871f-a4f598fc9813 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.147887 Sep 18 06:26:23.982545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 138/272] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:23 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:23.985451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:23.986065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:23.992513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:23.993107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:23.993425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:23.994118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:23.994572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:23.995197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:24.146241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:24.150281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:24.155678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:24.157119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:24.158231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-00cbcc8d-8858-48e1-b527-3b9b640b4faa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.176926 Sep 18 06:26:24.158808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 135/273] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:23 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:26:24.641268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:24.641868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:24.648240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:24.648712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:24.649139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:24.649815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:24.650264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:24.650708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:24.668629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:24.676509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:24.677212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:24.685057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:24.685474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:24.685912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:24.686618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:24.687073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:24.687530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:24.714906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:24.723504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a4307e67-2366-41a9-a0a3-e2b45a8c9116 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1429 microversion: 2.1 time: 0.085922 Sep 18 06:26:24.724078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 139/274] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:24 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1429 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:24.819030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:24.826606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1e7f05c3-da57-414f-b2ad-bd9138c4b5c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.154451 Sep 18 06:26:24.827167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 136/275] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:24 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:25.005108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:25.005671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:25.011849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.012328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.012954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.013432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.013881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.014299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.148715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:25.156233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5b242954-3d46-4bb7-8c3a-4834d61b7e5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.155121 Sep 18 06:26:25.156788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 140/276] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:25 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:25.183026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:25.183640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:25.189485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.189961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.190397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.191104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.191556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.191984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.342987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:25.353839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:25.357629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:25.360585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-86910603-c55a-44ff-b46a-d8446ce7c5a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.181429 Sep 18 06:26:25.376714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 137/277] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:25 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:25.747718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:25.748314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:25.755088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.755565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.755997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.757789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.758408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.761043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.801499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.802150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.802885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.803486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lazy-loading 'fault' on Instance uuid 4ab927d7-e82e-4cd4-a68d-a0895accce99 {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:26:25.860624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:25.861264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:25.874374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:25.876238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.876738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.879967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.885108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:25.885234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3adae824-b9a1-48a1-baef-7a82af35bbfe tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1305 microversion: 2.1 time: 0.141214 Sep 18 06:26:25.885988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:25.887121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:25.899790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 141/278] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:25 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1305 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:26.135858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:26.143792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4c8e780c-496c-4ab5-96b8-b21f52be417b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.297214 Sep 18 06:26:26.144366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 138/279] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:25 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:26.178833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:26.179443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:26.185530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:26.186048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:26.186482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:26.187233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:26.187681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:26.188098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:26.327330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:26.334910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-67bf1672-9d59-44de-b4df-f4e6ea207a65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.159952 Sep 18 06:26:26.335491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 142/280] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:26 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:26.388921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:26.389532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:26.395637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:26.396121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:26.396556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:26.397241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:26.397719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:26.398138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:26.538498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:26.546794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:26.548261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:26.549412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c389fe45-5660-4c39-8bf2-22c841def1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.164567 Sep 18 06:26:26.549968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 139/281] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:26 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:26.912793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:26.915201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:26.921746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:26.922228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:26.922687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:26.923380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:26.923835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:26.924259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:26.956198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:26.956680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:26.957111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:26.957551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lazy-loading 'fault' on Instance uuid 4ab927d7-e82e-4cd4-a68d-a0895accce99 {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:26:26.998642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:27.006378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-08fe01cf-11e2-4edd-85b9-d8e2bbaf7b12 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1305 microversion: 2.1 time: 0.101022 Sep 18 06:26:27.007237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 143/282] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:26 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1305 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:27.170714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:27.171952 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:27.181616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:27.182366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:27.183337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:27.184280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:27.185309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:27.186023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:27.359511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:27.361643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:27.368767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:27.369340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:27.369974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:27.370724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:27.371180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:27.371605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:27.397134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:27.405273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-28c7d3fe-a5e0-422a-9e2b-1fe7dc4e3f2d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.239144 Sep 18 06:26:27.406036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 140/283] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:27 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:27.520426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:27.528484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-146ea5ee-0519-4d02-87c0-36b06e35d5e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.174524 Sep 18 06:26:27.529159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 144/284] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:27 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:27.574948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:27.575557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:27.577367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:27.582482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:27.582998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:27.583436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:27.584178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:27.584682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:27.585719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:27.716846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:27.724814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:27.726265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:27.727401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f6b223c1-88d7-4321-8b79-9e8f9e523b29 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.156466 Sep 18 06:26:27.727957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 141/285] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:27 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:28.029300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:28.029827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:28.036237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.036749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.037191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.037877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.038331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.038799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.069107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.070291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.071265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.071265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lazy-loading 'fault' on Instance uuid 4ab927d7-e82e-4cd4-a68d-a0895accce99 {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:26:28.142836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d1f91620b8e4d87b7ce4a015b0d447d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89b44de54291465a8b30343a11d72f78', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:28.151031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a89dd175-3630-4231-9677-3e2ee23c758d tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 200 len: 1305 microversion: 2.1 time: 0.125553 Sep 18 06:26:28.152088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 145/286] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:28 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 1305 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:28.429378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:28.429992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:28.436941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.437430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.437976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.438808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.439261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.439690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.552297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:28.560061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:28.566129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.566637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.567278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.570991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.571467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.571902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.588523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:28.596438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dfcf0e27-5f69-43e8-bdbc-65412e93d527 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.171407 Sep 18 06:26:28.597004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 142/287] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:28 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:28.713234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:28.721089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a7ab1cea-6048-4606-a6a7-2ea7aaead194 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.172703 Sep 18 06:26:28.721656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 146/288] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:28 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:28.751612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:28.752398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:28.759337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.759977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.760489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.761255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:28.761781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:28.762280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:28.907408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:28.917034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:28.918914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:28.920024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5dfb1bab-c565-4576-b759-cded2b3330d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.172567 Sep 18 06:26:28.920584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 143/289] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:28 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:29.177036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:29.177654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] [instance: 4ab927d7-e82e-4cd4-a68d-a0895accce99] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:29.184802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:29.185285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:29.185718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:29.186407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:29.186911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:29.187336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:29.203179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] HTTP exception thrown: Instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 could not be found. Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Request method failure captured: Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 HTTP/1.1 Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrnohqmj451eJUqM3fpo5QN5DByhYVMal6LOhX7roglqjcmkMg97zHnr4j2i2MLXYHz1B1nJ6ZcNa3F8Vk2LjF8fic27kVDpNck0KWzJUq5LVRDBTKME4FLNt4KUqJ-GeIHPBeMS5DNLhy92TCC9wEalpz9wTahpzH6taetFJ4Q9-2w Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 89b44de54291465a8b30343a11d72f78 Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesNegativeTest-149536885 Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:26:29.205329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/89b44de54291465a8b30343a11d72f78"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_89b44de54291465a8b30343a11d72f78"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/89b44de54291465a8b30343a11d72f78"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/89b44de54291465a8b30343a11d72f78"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesNegativeTest-149536885 Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 89b44de54291465a8b30343a11d72f78 Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesNegativeTest-149536885 Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesNegativeTest-149536885-project Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 4d1f91620b8e4d87b7ce4a015b0d447d Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesNegativeTest-149536885-project Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 could not be found. Sep 18 06:26:29.206893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '4ab927d7-e82e-4cd4-a68d-a0895accce99'} Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 could not be found. Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:26:29.207664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 could not be found. Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 could not be found. Sep 18 06:26:29.209528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:29.210798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] Returning 404 to user: Instance 4ab927d7-e82e-4cd4-a68d-a0895accce99 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:26:29.210798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9fc8c87d-7ee4-4f53-912b-383a08d803f4 tempest-VolumesNegativeTest-149536885 tempest-VolumesNegativeTest-149536885-project] 10.222.0.20 "GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99" status: 404 len: 111 microversion: 2.1 time: 0.033921 Sep 18 06:26:29.210798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 147/290] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:29 2022] GET /compute/v2.1/servers/4ab927d7-e82e-4cd4-a68d-a0895accce99 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:26:29.619708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:29.620337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:29.627064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:29.627541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:29.627974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:29.628657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:29.629101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:29.629521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:29.746726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:29.747331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:29.753290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:29.753755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:29.754249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:29.754898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:29.756208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:29.756952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:29.762083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:29.771614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2fe6d79e-a07c-4906-be4b-70255f1ed721 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.156181 Sep 18 06:26:29.777775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 144/291] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:29 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:29.888977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:29.896925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1089cad4-fc37-412c-b6ee-caa15a2b26a4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.155340 Sep 18 06:26:29.897604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 148/292] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:29 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:29.945266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:29.945868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:29.951781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:29.952268 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:29.952707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:29.953402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:29.953849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:29.954332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:30.090380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:30.098975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:30.100517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:30.101782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b8c29be6-c40e-44c6-bc93-9f99efb710df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.160441 Sep 18 06:26:30.102336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 145/293] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:29 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:30.614559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-fd044680-e53d-4c36-8a2d-7b246b551e94 req-290860d5-f717-483f-8551-2426aaa26998 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ed3cfa3-6093-4c26-9877-01470acd2e16", "name": "network-vif-plugged", "status": "completed", "tag": "84893f57-6d56-4075-9e20-5bd22c657e44"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:30.624832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fd044680-e53d-4c36-8a2d-7b246b551e94 req-290860d5-f717-483f-8551-2426aaa26998 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:30.625635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fd044680-e53d-4c36-8a2d-7b246b551e94 req-290860d5-f717-483f-8551-2426aaa26998 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:30.626391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fd044680-e53d-4c36-8a2d-7b246b551e94 req-290860d5-f717-483f-8551-2426aaa26998 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:30.639214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-fd044680-e53d-4c36-8a2d-7b246b551e94 req-290860d5-f717-483f-8551-2426aaa26998 service nova] Creating event network-vif-plugged:84893f57-6d56-4075-9e20-5bd22c657e44 for instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 on hv-cinder-85535 Sep 18 06:26:30.644666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-fd044680-e53d-4c36-8a2d-7b246b551e94 req-290860d5-f717-483f-8551-2426aaa26998 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033697 Sep 18 06:26:30.646492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 149/294] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:30.796908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:30.798712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:30.827404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:30.827958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:30.828357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:30.831910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:30.832381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:30.832865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:30.920739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:30.921445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:30.932083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:30.932574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:30.933014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:30.933714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:30.934161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:30.934620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.100306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:31.110976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a6ff654a-185c-4af1-8b05-a0bc9d1f462b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.316409 Sep 18 06:26:31.127551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 146/295] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:30 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:31.142609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:31.144597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:31.156333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:31.157216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:31.158468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.160106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:31.160634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:31.161228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.182710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:31.196640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6f359cf5-5be9-4cf9-9ca7-2306a11d874c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1502 microversion: 2.1 time: 0.278017 Sep 18 06:26:31.198665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 150/296] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:30 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1502 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:31.222403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ef18efd2-286b-47b7-9748-9506294b7c8e"}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:31.225815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:31.236829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:31.240166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:31.240857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.243579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:31.246952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:31.247632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.358827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] HTTP exception thrown: Cannot 'attach_volume' instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb while it is in vm_state rescued Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Request method failure captured: Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: POST /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb/os-volume_attachments HTTP/1.1 Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 74 Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmU55Wg3gBiYn2aJSEvLyVJMV1h4CCP-z1W3AffyLY3fsW3JQ8OjuZv4iFhhTSFmKumF1s37EILY-hv50aLHoH2J1-083C3bmZIbw6-XE242ooMA_PlrxaMx2_DE94obNE1dW82mTeLq0C4XT8RYmdFLgmo0_0kmRhbQKjKMrBjrZ0 Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:26:31.372707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_91ef3478fd574180abe99393ba769124"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/91ef3478fd574180abe99393ba769124"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 7a6b8fbd31464e4d99449c74b4d1dda5 Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {"volumeAttachment": {"volumeId": "ef18efd2-286b-47b7-9748-9506294b7c8e"}} Sep 18 06:26:31.374104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Cannot 'attach_volume' instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb while it is in vm_state rescued Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': '1dd95cf5-d6be-4de9-9941-8b1d35855bcb', 'body': {'volumeAttachment': {'volumeId': 'ef18efd2-286b-47b7-9748-9506294b7c8e'}}} Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:26:31.374885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb while it is in vm_state rescued Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:26:31.376016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:26:31.379383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:26:31.379383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:26:31.379383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb while it is in vm_state rescued Sep 18 06:26:31.379383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:26:31.379383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Returning 409 to user: Cannot 'attach_volume' instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb while it is in vm_state rescued {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:26:31.379383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7ec94eee-e486-4bbd-9402-78cea8572a5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.156644 Sep 18 06:26:31.379383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 151/297] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:26:31 2022] POST /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb/os-volume_attachments => generated 152 bytes in 160 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 18 06:26:31.387476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:31.388007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:31.416598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:31.417151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:31.417591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.418312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:31.422611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:31.423640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.472674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:31.511602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:31.513284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:31.514715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-43559307-f231-4c07-ab56-ae7faee2b05f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.379034 Sep 18 06:26:31.515871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 147/298] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:31 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:31.574180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8720a264-14c2-473c-9c5c-d06fb020a981 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb/action" status: 202 len: 0 microversion: 2.1 time: 0.191380 Sep 18 06:26:31.575143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 152/299] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:26:31 2022] POST /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb/action => generated 0 bytes in 193 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:26:31.584343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:31.585735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:31.593810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:31.594705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:31.595658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.596751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:31.598528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:31.599718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:31.789119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:31.798618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-886331df-5f71-4809-826d-8a38bd096d2b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1510 microversion: 2.1 time: 0.218451 Sep 18 06:26:31.800387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 148/300] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:31 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1510 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:32.136819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:32.137689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:32.151191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:32.151703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:32.152512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:32.152854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:32.153307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:32.159529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:32.478545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:32.496006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-db141433-7a07-49bb-bd8d-aecc450405da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.364697 Sep 18 06:26:32.496585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 153/301] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:32 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:32.541259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:32.541883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:32.551536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:32.553173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:32.556398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:32.557346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:32.557979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:32.559703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:32.739141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-32c72343-0b3a-45c7-bc19-4386d584aa11 req-77915fb5-f30f-48c6-ad44-6ec2c41ae6b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b4bf8fc2-b57b-4198-939c-bd88739c785e", "name": "network-vif-plugged", "status": "completed", "tag": "1956a12e-1f36-454a-baa5-f949db978b94"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:32.751252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-32c72343-0b3a-45c7-bc19-4386d584aa11 req-77915fb5-f30f-48c6-ad44-6ec2c41ae6b6 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:32.753568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-32c72343-0b3a-45c7-bc19-4386d584aa11 req-77915fb5-f30f-48c6-ad44-6ec2c41ae6b6 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:32.755230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-32c72343-0b3a-45c7-bc19-4386d584aa11 req-77915fb5-f30f-48c6-ad44-6ec2c41ae6b6 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:32.794701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-32c72343-0b3a-45c7-bc19-4386d584aa11 req-77915fb5-f30f-48c6-ad44-6ec2c41ae6b6 service nova] Creating event network-vif-plugged:1956a12e-1f36-454a-baa5-f949db978b94 for instance b4bf8fc2-b57b-4198-939c-bd88739c785e on hv-cinder-85535 Sep 18 06:26:32.807190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-32c72343-0b3a-45c7-bc19-4386d584aa11 req-77915fb5-f30f-48c6-ad44-6ec2c41ae6b6 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071691 Sep 18 06:26:32.810887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 154/302] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:32.828695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-953dbbf8-9479-4e67-8bf7-655486007c83 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:32.841960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-953dbbf8-9479-4e67-8bf7-655486007c83 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:32.842337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-953dbbf8-9479-4e67-8bf7-655486007c83 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:32.842810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-953dbbf8-9479-4e67-8bf7-655486007c83 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:32.924686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:32.937752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:32.939230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:32.940294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6cbfb798-fac8-488f-b2d0-0cfadef81086 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1407 microversion: 2.20 time: 0.402961 Sep 18 06:26:32.941150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 149/303] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:32 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1407 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:32.944897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-953dbbf8-9479-4e67-8bf7-655486007c83 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Going to try to stop instance {{(pid=120381) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 18 06:26:32.946664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:32.947688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:32.954073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:32.955216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:32.955821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:32.956822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:32.957722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:32.958306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:33.006034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:33.052443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-953dbbf8-9479-4e67-8bf7-655486007c83 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/action" status: 202 len: 0 microversion: 2.1 time: 0.230277 Sep 18 06:26:33.054016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 155/304] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:26:32 2022] POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/action => generated 0 bytes in 242 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:26:33.083655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:33.085669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:33.100062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:33.100062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:33.112532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:33.117710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:33.118935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:33.122882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:33.183705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:33.195578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-53f56104-7351-47ef-9246-bd2e72f404ac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1510 microversion: 2.1 time: 0.253121 Sep 18 06:26:33.200527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 150/305] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:32 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1510 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:33.347487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:33.356433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b82aff8a-ba27-4651-a093-940133832f0e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.297917 Sep 18 06:26:33.357086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 156/306] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:33 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:33.519748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:33.520609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:33.532884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:33.533388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:33.533830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:33.534542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:33.536285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:33.537026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:33.684431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:33.692854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-57b1ed91-b7f0-4c10-9507-3b776b03f092 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.177153 Sep 18 06:26:33.705153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 151/307] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:33 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:33.971004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:33.971857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:33.978822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:33.979944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:33.980620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:33.982626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:33.983098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:33.983528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:34.178122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:34.189993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:34.192945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:34.195128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eda73f51-1b74-4d5f-8655-63ed233d523c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1599 microversion: 2.20 time: 0.228373 Sep 18 06:26:34.202970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 157/308] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:26:33 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1599 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:26:34.227888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:34.229516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:34.235561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:34.237621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:34.237756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:34.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:34.239168 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:34.240562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:34.241957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:34.242630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:34.247280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:34.247809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:34.248316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:34.249223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:34.250206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:34.250972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:34.274719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:34.428211 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-33800a05-c49b-4bf2-ab23-ba78aba0c2c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.207819 Sep 18 06:26:34.429459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 152/309] 10.222.0.20 () {60 vars in 1316 bytes} [Sun Sep 18 06:26:34 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-interface => generated 284 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:26:34.441228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:34.442351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:34.454254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:34.455541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:34.456246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:34.457176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:34.457849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:34.458503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:34.547127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:34.559274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-156e362d-0c14-45b3-a0b8-739cef20aa71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1510 microversion: 2.1 time: 0.331247 Sep 18 06:26:34.561956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 158/310] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:34 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1510 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:34.717675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:34.718616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:34.725225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:34.725963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:34.726543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:34.727584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:34.728190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:34.728771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:35.043187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:35.062985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-62cd6118-a99c-4f73-9d4c-05d8d2b4c60c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.631898 Sep 18 06:26:35.063566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 153/311] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:34 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:35.149365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:35.157624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-18de02f5-ba79-4485-bf0b-58373f2a5fd9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.443754 Sep 18 06:26:35.170892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 159/312] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:34 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:35.583929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:35.585606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:35.599589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:35.600913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:35.601517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:35.602411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:35.602886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:35.603465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:35.741794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-f7c1a406-d64f-44a4-a090-c54670463c5f req-6f647306-4220-4e7d-8e68-3eb175361d57 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5ed3cfa3-6093-4c26-9877-01470acd2e16", "tag": "84893f57-6d56-4075-9e20-5bd22c657e44"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:35.752664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-f7c1a406-d64f-44a4-a090-c54670463c5f req-6f647306-4220-4e7d-8e68-3eb175361d57 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:35.753227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-f7c1a406-d64f-44a4-a090-c54670463c5f req-6f647306-4220-4e7d-8e68-3eb175361d57 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:35.753725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-f7c1a406-d64f-44a4-a090-c54670463c5f req-6f647306-4220-4e7d-8e68-3eb175361d57 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:35.766563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-f7c1a406-d64f-44a4-a090-c54670463c5f req-6f647306-4220-4e7d-8e68-3eb175361d57 service nova] Creating event network-changed:84893f57-6d56-4075-9e20-5bd22c657e44 for instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 on hv-cinder-85535 Sep 18 06:26:35.775151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-f7c1a406-d64f-44a4-a090-c54670463c5f req-6f647306-4220-4e7d-8e68-3eb175361d57 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034796 Sep 18 06:26:35.775868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 160/313] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:35.811490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:35.826526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-457ecc06-bbd1-4a34-9a9a-f07a0823e950 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1510 microversion: 2.1 time: 0.246748 Sep 18 06:26:35.838860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 154/314] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:35 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1510 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:36.086628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:36.090606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:36.105189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:36.105935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:36.106521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:36.107576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:36.108741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:36.110882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:36.180790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:36.182537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:36.191293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:36.192122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:36.192861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:36.193681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:36.194517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:36.195209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:36.420798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:36.431652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2b41bcf5-313b-4d98-9776-8f54d26bd5b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.349924 Sep 18 06:26:36.434481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 161/315] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:36 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:36.527266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:36.546429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-35d72e67-207d-4117-aaac-03fb5b8f197e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1359 microversion: 2.1 time: 0.369075 Sep 18 06:26:36.548040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 155/316] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:36 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1359 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:36.854783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:36.857010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:36.873583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:36.874106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:36.876533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:36.877295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:36.877967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:36.880282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:37.206467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:37.219546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bd6a5198-70db-4f9c-84ef-5f6b4b5b75ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1516 microversion: 2.1 time: 0.369753 Sep 18 06:26:37.220763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 162/317] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:36 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1516 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:37.459889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:37.461475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:37.479102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:37.480405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:37.481410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:37.482818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:37.485618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:37.486234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:37.571605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:37.572327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:37.584378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:37.585345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:37.585993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:37.590170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:37.590962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:37.592902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:38.180661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:38.201369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3f7772c7-a9cc-4caa-9494-fd212eb18c54 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.746033 Sep 18 06:26:38.202484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 156/318] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:37 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:38.279169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:38.292610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-238f6b02-cc08-44f1-98fd-56e520df4be0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1550 microversion: 2.1 time: 0.726446 Sep 18 06:26:38.293559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 163/319] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:37 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1550 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:38.321351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:38.322435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:38.334932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:38.335445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:38.341766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:38.342977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:38.344819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:38.359686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:38.661523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:38.672578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bf88c9f3-cb1d-47d0-bf7c-9cd011bb1b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1550 microversion: 2.1 time: 0.360429 Sep 18 06:26:38.673758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 157/320] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:38 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1550 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:38.697895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:38.703459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:38.708230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Searching by: {'deleted': False, 'project_id': '2af8940da97b4298a92e1d45f70cfe29'} {{(pid=120381) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 18 06:26:38.722651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:38.723129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:38.727348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:38.730968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:38.731897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:38.735528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:38.842128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.multi_cell_list [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120381) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 18 06:26:38.877814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8ee47618-c1d5-4af0-95f7-816f73291f39 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'create', calling method: .version_select at 0x7fe671c86040>, body: {"keypair": {"name": "tempest-keypair-164943999", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCVN1O8DQpVdTGK8+79afGlBYJFn2/NWRiHiHXtvBdjmUhmH7xjSvCpquTY61jtBPSQ1Vff46wySEG6InbMRl1Qj82dYY6KyVk7ySMVAk/03jf9oXyJ6uxOw2ohR0eieKQ=="}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:38.906037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-8ee47618-c1d5-4af0-95f7-816f73291f39 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Getting quotas for project 91ef3478fd574180abe99393ba769124. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:38.912998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-8ee47618-c1d5-4af0-95f7-816f73291f39 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Getting quotas for user 7a6b8fbd31464e4d99449c74b4d1dda5 and project 91ef3478fd574180abe99393ba769124. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:38.952075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8ee47618-c1d5-4af0-95f7-816f73291f39 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.078094 Sep 18 06:26:38.952694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 158/321] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:26:38 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:39.008411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aa2af458-6b1d-4f31-bdd6-f706caed3968 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.315328 Sep 18 06:26:39.010807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 164/322] 10.222.0.20 () {58 vars in 1125 bytes} [Sun Sep 18 06:26:38 2022] GET /compute/v2.1/servers => generated 343 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:39.024716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:39.025591 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:39.039389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:39.039885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:39.048366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:39.049141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:39.049612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:39.050036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:39.230097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:39.231596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:39.244771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:39.245559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:39.246432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:39.247721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:39.250052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:39.251862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:39.298072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:39.309789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-69edd069-b01d-4ce1-b9b8-b928b6ffc33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1550 microversion: 2.1 time: 0.292395 Sep 18 06:26:39.334459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 159/323] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:39 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1550 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:39.515775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:39.527696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eefe692f-ba05-48ce-a9b1-2383112a197d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.303680 Sep 18 06:26:39.550046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 165/324] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:39 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:39.635420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1560287161", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "networks": [{"uuid": "3fc1a2fb-9883-4c17-86ff-23fe397da81c"}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:39.701739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f6262f0d-3a2c-43cb-a26f-f234615e29e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: .version_select at 0x7fe6704c80d0>, body: {"keypair": {"name": "tempest-keypair-143579324", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIutGOgeqPerrY9gz8ld3dOejvYSk2n1LEWl0g3v73adIU4BAoC8sywTkiLAnHgLPnxyJ37uaJu4WTWOYP/13Q5y0ApVSFuJqgqeUP2idycPkCnsSmo8L0nAkPMRW7E53g=="}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:39.730340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-f6262f0d-3a2c-43cb-a26f-f234615e29e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:39.736869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-f6262f0d-3a2c-43cb-a26f-f234615e29e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for user d85160b01d824bc3a32b425fab98d2c7 and project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:39.753408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:39.779704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] validate_networks() for [('3fc1a2fb-9883-4c17-86ff-23fe397da81c', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:26:39.780758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f6262f0d-3a2c-43cb-a26f-f234615e29e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.088142 Sep 18 06:26:39.782901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 166/325] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:26:39 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:40.554278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:40.555103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:40.562920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:40.566317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:40.567094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:40.568518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:40.569309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:40.570141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:40.665250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:26:40.665667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:26:40.666144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:26:40.666557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:26:40.666996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:26:40.684870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:40.687778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:40.689201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:40.693120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:40.694021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:40.698317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:40.742627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Getting quotas for project 1d431bdc9a2244dcb1bb589dee792d02. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:40.747312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Getting quotas for user e702f6ad9e38492497e0ae33abb9aae7 and project 1d431bdc9a2244dcb1bb589dee792d02. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:40.777752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:26:40.780231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:26:40.882456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-60df6a35-e369-4f65-9dbd-14bd75cb11f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.252483 Sep 18 06:26:40.899571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 160/326] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:26:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 1270 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:26:40.904496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-03861bf7-45f5-45a3-9902-9265c366f8c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:40.905117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-03861bf7-45f5-45a3-9902-9265c366f8c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:41.017135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-03861bf7-45f5-45a3-9902-9265c366f8c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:41.031324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-03861bf7-45f5-45a3-9902-9265c366f8c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1219 microversion: 2.1 time: 0.130503 Sep 18 06:26:41.031840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 161/327] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:40 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1219 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:41.284471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "466d74bb-d7b4-47ff-a708-ab67d2eec121"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:41.287046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:41.302445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:41.303045 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:41.303592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:41.304376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:41.304839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:41.305352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:41.401049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:41.415859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5cb6a603-973f-47d0-8cbd-cfa0c5f5f91c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.866651 Sep 18 06:26:41.418273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 167/328] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:40 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 870 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:41.914310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5e23f832-a25f-453d-907a-7a9613a6d020 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.614462 Sep 18 06:26:41.917042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 162/329] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:26:41 2022] POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 194 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:42.054725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:42.056395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:42.078504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:42.079263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:42.084175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:42.085558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:42.086512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:42.087794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:42.220462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:42.230347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-82ee38fa-7ced-4415-99c1-94d288e75f31 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1215 microversion: 2.1 time: 0.180702 Sep 18 06:26:42.231475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 168/330] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:42 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1215 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:42.444269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:42.445989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:42.466148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:42.467023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:42.468140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:42.470927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:42.472543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:42.474051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:42.788185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1431704852", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "0831a5b2-a7c5-4c5e-ba2b-086694d4e37b"}], "security_groups": [{"name": "tempest-securitygroup--1524457111"}], "key_name": "tempest-keypair-164943999", "user_data": "}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:42.824914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:42.839215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f47fe1c6-1615-40f2-894d-e6e35911b572 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.401223 Sep 18 06:26:42.856580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 163/331] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:42 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:42.967312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] validate_networks() for [('0831a5b2-a7c5-4c5e-ba2b-086694d4e37b', None, None, None, None, None)] {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:26:43.161458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1105524073", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d572beb8-e9a5-4e8d-a0f6-5ec49c88e496"}], "security_groups": [{"name": "tempest-securitygroup--1925438782"}], "key_name": "tempest-keypair-143579324", "user_data": "}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:43.409643 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] validate_networks() for [('d572beb8-e9a5-4e8d-a0f6-5ec49c88e496', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:26:44.379935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:26:44.381183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:26:44.382180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:26:44.382822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:26:44.385980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:26:44.400290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:44.401118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:44.401894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:44.402970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:44.403741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:44.404337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:44.442948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Getting quotas for project 91ef3478fd574180abe99393ba769124. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:44.456646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Getting quotas for user 7a6b8fbd31464e4d99449c74b4d1dda5 and project 91ef3478fd574180abe99393ba769124. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:44.498700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:26:44.504392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:26:44.645222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8bb75902-6517-461b-a8ba-760ad2d67199 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.862512 Sep 18 06:26:44.649455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 169/332] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:26:42 2022] POST /compute/v2.1/servers => generated 409 bytes in 1868 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 06:26:44.659901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:44.669062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:44.690475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:44.691369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:44.692789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:44.695924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:44.698357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:44.700285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:44.805755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:26:44.806556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:26:44.807067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:26:44.807483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:26:44.808041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:26:44.829518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:44.835663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:44.836365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:44.838158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:44.838620 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:44.839059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:44.878664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:44.891114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-357f5a77-0ce0-4d03-bb7f-97378282f992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1277 microversion: 2.1 time: 0.240440 Sep 18 06:26:44.891697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 170/333] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:44 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1277 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:44.904492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:44.905818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:44.926081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:44.931731 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:44.934827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:44.937168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:44.943983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:44.944748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:44.947294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:44.966987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for user d85160b01d824bc3a32b425fab98d2c7 and project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:45.008365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:26:45.014240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:26:45.176948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-054f016b-6d55-4183-aedc-26dabba14117 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.028181 Sep 18 06:26:45.181501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 164/334] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:26:43 2022] POST /compute/v2.1/servers => generated 402 bytes in 2036 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:26:45.202664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-28947def-d8d3-4325-a799-d4dbba7dbf90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:45.205003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-28947def-d8d3-4325-a799-d4dbba7dbf90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:45.341842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:45.353492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c36e972f-28c7-4660-baa4-50de898dba1d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.457297 Sep 18 06:26:45.358897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 171/335] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:44 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:45.367288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6de569bf-4671-4da9-912c-7c30fe9cb970 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:45.368080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6de569bf-4671-4da9-912c-7c30fe9cb970 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:45.494875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6de569bf-4671-4da9-912c-7c30fe9cb970 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:45.495443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-28947def-d8d3-4325-a799-d4dbba7dbf90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:45.505145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6de569bf-4671-4da9-912c-7c30fe9cb970 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1241 microversion: 2.1 time: 0.144737 Sep 18 06:26:45.506077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 172/336] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:45 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1241 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:45.512680 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-28947def-d8d3-4325-a799-d4dbba7dbf90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1245 microversion: 2.1 time: 0.329920 Sep 18 06:26:45.513281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 165/337] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:45 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1245 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:45.921459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:45.922349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:45.936334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:45.936855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:45.938926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:45.939813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:45.943213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:45.947201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:46.337811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:46.353652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-906e4678-7f85-4142-b7ce-9f44ad32c81d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.438017 Sep 18 06:26:46.358242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 173/338] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:45 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:46.377612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:46.378961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:46.395316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:46.399357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:46.400249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:46.402356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:46.404262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:46.404940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:46.529683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:46.531012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:46.540978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:46.541477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:46.547856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:46.548777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:46.549632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:46.550136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:46.784079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:46.806976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-31218c84-29d6-4eb1-818e-e3df64c1cfe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1237 microversion: 2.1 time: 0.281509 Sep 18 06:26:46.828078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 174/339] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:46 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1237 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:46.837609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:46.838968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:46.845812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:46.846821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:46.847421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:46.848297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:46.848978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:46.849812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:46.857541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:47.078168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:47.092075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:47.102020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6316eb20-a234-4a01-9974-97ae1299386e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1297 microversion: 2.1 time: 0.272612 Sep 18 06:26:47.106111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 175/340] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1297 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:47.108289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e3297972-3912-4993-8ab2-2c4a3c575549 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.731986 Sep 18 06:26:47.124567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 166/341] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:46 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 751 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:47.378532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:47.379191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:47.412761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:47.413554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:47.416925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:47.418566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:47.419864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:47.420689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:47.776573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:47.785619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-74f5c353-13a8-479e-901f-fd5b9b256332 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.411934 Sep 18 06:26:47.786205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 176/342] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:47 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:47.837684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:47.838365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:47.854223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:47.855062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:47.855691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:47.856554 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:47.857423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:47.858042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:47.899559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-9877f725-49d1-47de-8831-9bc9673675c5 req-107a09bb-0cb1-4e41-9519-7aa31ca681fc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "141e3677-77e2-4021-918a-5fbe5292398e", "tag": "5a5c550c-fee5-4d01-961b-e15307eef48d"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:47.932525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-9877f725-49d1-47de-8831-9bc9673675c5 req-107a09bb-0cb1-4e41-9519-7aa31ca681fc service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:47.933261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-9877f725-49d1-47de-8831-9bc9673675c5 req-107a09bb-0cb1-4e41-9519-7aa31ca681fc service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:47.933904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-9877f725-49d1-47de-8831-9bc9673675c5 req-107a09bb-0cb1-4e41-9519-7aa31ca681fc service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:47.964652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-9877f725-49d1-47de-8831-9bc9673675c5 req-107a09bb-0cb1-4e41-9519-7aa31ca681fc service nova] Creating event network-changed:5a5c550c-fee5-4d01-961b-e15307eef48d for instance 141e3677-77e2-4021-918a-5fbe5292398e on hv-cinder-85535 Sep 18 06:26:47.979813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-9877f725-49d1-47de-8831-9bc9673675c5 req-107a09bb-0cb1-4e41-9519-7aa31ca681fc service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099218 Sep 18 06:26:47.980530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 177/343] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:48.011179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:48.023249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d678d54e-6624-4fbc-866a-70e4e8791edf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1293 microversion: 2.1 time: 0.190598 Sep 18 06:26:48.025363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 167/344] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:47 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1293 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:48.135253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:48.135422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:48.135887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:48.136018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:48.143806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:48.144558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:48.145285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:48.145970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:48.147544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:48.150112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:48.150272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:48.150601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:48.151124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:48.151331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:48.151958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:48.159605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:48.335542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:48.347724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-456d96c6-3a9b-4c0b-b45a-baa5bc7c6f5f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1303 microversion: 2.1 time: 0.218155 Sep 18 06:26:48.361058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 178/345] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1303 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:48.577196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:48.608766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-800860d0-39e8-4089-a004-80734e9a283c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1721 microversion: 2.1 time: 0.477533 Sep 18 06:26:48.610675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 168/346] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:48 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1721 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:48.810877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:48.811994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:48.818422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:48.819223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:48.819757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:48.820559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:48.821108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:48.821627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.049287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:49.050373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:49.053020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:49.064947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.069336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.069844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.071991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.072513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.074188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.120645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:49.132320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-133ab561-81bd-49d8-bf1a-475931ee106b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.325364 Sep 18 06:26:49.141406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 179/347] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:48 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:49.249496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:49.263985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b9cddb5a-1557-4bea-aebb-90198673031a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1293 microversion: 2.1 time: 0.219764 Sep 18 06:26:49.279516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 169/348] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:49 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1293 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:49.376857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:49.377628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:49.392973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.393528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.394081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.394970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.395538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.396156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.641160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:49.642172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:49.653042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.654806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.655489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.656383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.656987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.657565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.896540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:49.911406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9850238a-c6f2-41d8-b321-45e7001b4dc4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.538832 Sep 18 06:26:49.930609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 180/349] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:49.930927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:49.931063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:49.931063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:49.931575 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a479215d-35a2-4b01-b7c6-553235d491b5 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1698 microversion: 2.1 time: 0.292554 Sep 18 06:26:49.935059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 170/350] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:49 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1698 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:49.952089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.955048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.955698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.956515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.956973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2d392f9e-07be-4564-8784-7693658cb503 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:49.957079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.957403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:49.964253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d392f9e-07be-4564-8784-7693658cb503 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:49.964736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d392f9e-07be-4564-8784-7693658cb503 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:49.965368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d392f9e-07be-4564-8784-7693658cb503 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:50.032855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2d392f9e-07be-4564-8784-7693658cb503 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Going to try to start instance {{(pid=120380) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 18 06:26:50.075278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:50.104815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2d392f9e-07be-4564-8784-7693658cb503 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/action" status: 202 len: 0 microversion: 2.1 time: 0.150801 Sep 18 06:26:50.105033 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 171/351] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:26:49 2022] POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/action => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 06:26:50.116341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:50.117405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:50.128330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:50.129514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:50.133434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:50.139796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:50.140601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:50.143214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:50.325221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:50.335456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bf61dcdb-f826-4f06-91e4-07a374c9a89b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1596 microversion: 2.1 time: 0.417817 Sep 18 06:26:50.336189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 181/352] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:49 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1596 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:50.342363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:50.345480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:50.363345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:50.363864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:50.364367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:50.369252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:50.370513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:50.371395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:50.499187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:50.511609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-612ee1d7-b871-4e18-8ed8-1192831ae7b8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1707 microversion: 2.1 time: 0.401859 Sep 18 06:26:50.525888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 172/353] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:50 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1707 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:50.538926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:50.539699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:50.557122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:50.557622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:50.559966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:50.562356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:50.562864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:50.563299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:50.694245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:50.721328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d898a0ab-7d41-4f6b-b48d-f9b0ac111464 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1299 microversion: 2.1 time: 0.193997 Sep 18 06:26:50.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:50.722146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 173/354] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:50 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1299 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:50.737459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c34a07f6-89e6-4463-8a0a-a0ce9be9c71b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.399293 Sep 18 06:26:50.738370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 182/355] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:50 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:50.945756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:50.947188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:50.963463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:50.964269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:50.965175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:50.966355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:50.967121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:50.967706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:51.213580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:51.231906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-985536c9-2126-40b8-a6b5-263f648f4ab8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.287632 Sep 18 06:26:51.233285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 174/356] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:51.541972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:51.543334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:51.556339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:51.556844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:51.557290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:51.558038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:51.558472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:51.558930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:51.636609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-84856470-c869-4af9-a9b1-248a133fe9e7 req-2e8bdb01-806f-4557-b9e7-53432c900407 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a8254cee-e629-4d82-988c-708bac734c1e", "tag": "7ff7d3c9-95be-4eea-b0b3-a8bd5d358217"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:51.659741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-84856470-c869-4af9-a9b1-248a133fe9e7 req-2e8bdb01-806f-4557-b9e7-53432c900407 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:51.660362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-84856470-c869-4af9-a9b1-248a133fe9e7 req-2e8bdb01-806f-4557-b9e7-53432c900407 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:51.663751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-84856470-c869-4af9-a9b1-248a133fe9e7 req-2e8bdb01-806f-4557-b9e7-53432c900407 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:51.680545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-84856470-c869-4af9-a9b1-248a133fe9e7 req-2e8bdb01-806f-4557-b9e7-53432c900407 service nova] Creating event network-changed:7ff7d3c9-95be-4eea-b0b3-a8bd5d358217 for instance a8254cee-e629-4d82-988c-708bac734c1e on hv-cinder-85535 Sep 18 06:26:51.694787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-84856470-c869-4af9-a9b1-248a133fe9e7 req-2e8bdb01-806f-4557-b9e7-53432c900407 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061956 Sep 18 06:26:51.699321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 175/357] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:51.999696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4364ec75-a2f0-4acb-996c-0cdb2dcadeaa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Action: 'create', calling method: .version_select at 0x7fe670600160>, body: {"keypair": {"name": "tempest-TestShelveInstance-1890991792", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIA+ScUgQFMDHRzqsO22cdEOK9vxN3oYdwrplyaxFcAM1DeJkuRGNgkckq8BRFlaZgAcLtVv4dQKlJ4KVXlyWw1mtO7dxzbeIs8U3dsDe1+rdIN8ZGZvU+fZLyszRpmGlA=="}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:52.019349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-4364ec75-a2f0-4acb-996c-0cdb2dcadeaa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Getting quotas for project d0adbb8b987449f08ee39d325424b1da. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:26:52.024560 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-4364ec75-a2f0-4acb-996c-0cdb2dcadeaa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Getting quotas for user c2b511ca42e6427aa2f1627c87ad3fad and project d0adbb8b987449f08ee39d325424b1da. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:26:52.056488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4364ec75-a2f0-4acb-996c-0cdb2dcadeaa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.311994 Sep 18 06:26:52.057133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 176/358] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:26:51 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:52.062152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:52.063271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:52.071017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:52.071805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:52.072469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:52.073415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:52.074125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:52.074943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:52.143015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:52.154886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e9597254-a65f-4354-a65c-e5479998078d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1707 microversion: 2.1 time: 0.618040 Sep 18 06:26:52.171815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 183/359] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:51 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1707 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:52.181652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:52.184276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:52.191420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:52.192379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:52.193473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:52.194175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:52.194819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:52.195390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:52.203752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:52.221533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bb4ba61a-fba1-4896-b2e6-5d8619e24765 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1299 microversion: 2.1 time: 0.162886 Sep 18 06:26:52.223156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 177/360] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:52 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1299 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:52.257133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:52.257757 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:52.265658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:52.266159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:52.266616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:52.274920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:52.276486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:52.277152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:52.798117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:52.843738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e4475672-43fb-483d-8d88-63e42a598932 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.658672 Sep 18 06:26:52.847238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 184/361] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:52 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:52.940407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:52.950886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d4db10c9-8e27-49c6-a581-799427e119f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.698563 Sep 18 06:26:52.967073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 178/362] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:52 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:53.179008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:53.179625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:53.187158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:53.196047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:53.196923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:53.198050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:53.199105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:53.201020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:53.248136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:53.249354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:53.260839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:53.262733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:53.263354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:53.266618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:53.267239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:53.267667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:53.553170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:53.571765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-789cb594-d1fe-4b46-b836-fd9f4ab1a520 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1707 microversion: 2.1 time: 0.397132 Sep 18 06:26:53.572553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 185/363] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:53 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1707 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:53.610696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:53.619312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c352065f-d2e0-419d-ae2d-2b4483d9cb27 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.375578 Sep 18 06:26:53.620321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 179/364] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:53 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:53.869904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:53.870686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:53.878636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:53.886740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:53.891058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:53.892139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:53.892944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:53.893591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:53.981498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:53.982452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:54.001826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:54.002737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:54.003426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:54.004395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:54.005104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:54.006990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:54.429266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:54.657428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:54.669262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-19d28618-ce76-4019-8424-491ea6bbe14e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.803677 Sep 18 06:26:54.669869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 186/365] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:53 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 805 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:54.679637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:54.679637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:54.693119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:54.693277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:54.693738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:54.694095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:54.703013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:54.703616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:54.704195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:54.713364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9d6e9fda-00ae-48b8-ae35-724e5e3e2c74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.734530 Sep 18 06:26:54.714919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 180/366] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:54.729721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:54.730567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:54.747162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:54.747661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:54.748095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:54.748789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:54.753658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:54.754278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:54.858429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:55.048037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:55.067540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-51d74e5c-fd90-4875-a498-d43b66d92fb9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1707 microversion: 2.1 time: 0.396404 Sep 18 06:26:55.068633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 187/367] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:54 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1707 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:55.074890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-069fc55e-8ab0-4d46-93e7-d2e74d68e020 req-563e89ed-ab3b-46ab-92ca-90fe713f6867 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b4bf8fc2-b57b-4198-939c-bd88739c785e", "tag": "1956a12e-1f36-454a-baa5-f949db978b94"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:55.092271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-069fc55e-8ab0-4d46-93e7-d2e74d68e020 req-563e89ed-ab3b-46ab-92ca-90fe713f6867 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:55.093123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-069fc55e-8ab0-4d46-93e7-d2e74d68e020 req-563e89ed-ab3b-46ab-92ca-90fe713f6867 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:55.093728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-069fc55e-8ab0-4d46-93e7-d2e74d68e020 req-563e89ed-ab3b-46ab-92ca-90fe713f6867 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:55.108627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-069fc55e-8ab0-4d46-93e7-d2e74d68e020 req-563e89ed-ab3b-46ab-92ca-90fe713f6867 service nova] Creating event network-changed:1956a12e-1f36-454a-baa5-f949db978b94 for instance b4bf8fc2-b57b-4198-939c-bd88739c785e on hv-cinder-85535 Sep 18 06:26:55.115194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-069fc55e-8ab0-4d46-93e7-d2e74d68e020 req-563e89ed-ab3b-46ab-92ca-90fe713f6867 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044476 Sep 18 06:26:55.116608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 188/368] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:55.282597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:55.299519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-df83d912-e17a-4f9f-825d-1e38acb32f99 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.582983 Sep 18 06:26:55.300719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 181/369] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:54 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:55.658357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=12 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:55.665619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-7eebc794-6a0b-4949-8512-736125bedf23 req-d2b93822-f074-4dc2-a835-280039f22265 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2131eb93-ac09-4fc4-991e-3a0cc04b5bbd", "tag": "96889ca2-75ee-41de-b4a7-a0b675cf59ff"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:55.687723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-7eebc794-6a0b-4949-8512-736125bedf23 req-d2b93822-f074-4dc2-a835-280039f22265 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:55.688041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-7eebc794-6a0b-4949-8512-736125bedf23 req-d2b93822-f074-4dc2-a835-280039f22265 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:55.692227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-7eebc794-6a0b-4949-8512-736125bedf23 req-d2b93822-f074-4dc2-a835-280039f22265 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:55.703696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:55.704384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:55.727953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:55.728909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:55.729627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:55.730952 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:55.731877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:55.732494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:55.737864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-7eebc794-6a0b-4949-8512-736125bedf23 req-d2b93822-f074-4dc2-a835-280039f22265 service nova] Creating event network-changed:96889ca2-75ee-41de-b4a7-a0b675cf59ff for instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd on hv-cinder-85535 Sep 18 06:26:55.755946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-7eebc794-6a0b-4949-8512-736125bedf23 req-d2b93822-f074-4dc2-a835-280039f22265 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.097506 Sep 18 06:26:55.759460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 189/370] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:55.774341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-921cd909-9579-4b0f-bc25-fc85ddb459d8 req-0c9f2d57-4652-416e-ab61-e5a47b7c2cfc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "141e3677-77e2-4021-918a-5fbe5292398e", "name": "network-vif-plugged", "status": "completed", "tag": "5a5c550c-fee5-4d01-961b-e15307eef48d"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:55.799354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-921cd909-9579-4b0f-bc25-fc85ddb459d8 req-0c9f2d57-4652-416e-ab61-e5a47b7c2cfc service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:55.801987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-921cd909-9579-4b0f-bc25-fc85ddb459d8 req-0c9f2d57-4652-416e-ab61-e5a47b7c2cfc service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:55.802690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-921cd909-9579-4b0f-bc25-fc85ddb459d8 req-0c9f2d57-4652-416e-ab61-e5a47b7c2cfc service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:55.819346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-921cd909-9579-4b0f-bc25-fc85ddb459d8 req-0c9f2d57-4652-416e-ab61-e5a47b7c2cfc service nova] Creating event network-vif-plugged:5a5c550c-fee5-4d01-961b-e15307eef48d for instance 141e3677-77e2-4021-918a-5fbe5292398e on hv-cinder-85535 Sep 18 06:26:55.826028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-921cd909-9579-4b0f-bc25-fc85ddb459d8 req-0c9f2d57-4652-416e-ab61-e5a47b7c2cfc service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064964 Sep 18 06:26:55.827018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 190/371] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:55.832693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:55.833929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:55.841370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:55.842291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:55.842979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:55.843924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:55.844545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:55.845273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:56.282668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:56.294973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:56.299703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e15a4889-c76c-42d0-b17d-84eb9998118a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.600815 Sep 18 06:26:56.309122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7196ada6-f2ba-4caa-b300-4b623d91bf1b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.480585 Sep 18 06:26:56.310243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 191/372] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:56.315843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 182/373] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:55 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:26:56.321695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:56.327688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:56.327950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:56.337059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:56.337638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:56.338090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:56.338505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:56.338860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:56.339323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:56.339756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:56.347530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:56.352510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:56.353993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:56.359485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:56.360093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:56.360651 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:56.771244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:56.783616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6ff9a460-0f08-477c-8b2b-7845e984d5d6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1711 microversion: 2.1 time: 0.471402 Sep 18 06:26:56.784525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 192/374] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:56 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1711 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:56.804203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:56.815040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fee7d25c-a844-417d-a17a-af74803080f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.493542 Sep 18 06:26:56.817010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 183/375] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:56 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:57.296873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1324081990"}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:57.297580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:57.305359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:57.305846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:57.306350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:57.307096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:57.307580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:57.308004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:57.328515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:57.329212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:57.336302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:57.336865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:57.337384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:57.338165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:57.338791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:57.339451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:57.518331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.network.security_group_api [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Adding security group ec5d246c-cc78-42f2-9d89-dd53ee15f560 to port 1956a12e-1f36-454a-baa5-f949db978b94 Sep 18 06:26:57.610688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:57.622378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-37e2a56a-5d21-4395-b87b-a46c07411f3a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.297450 Sep 18 06:26:57.627202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 184/376] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:57 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:57.638002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:57.640494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:57.648574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:57.660315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:57.661891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:57.662865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:57.663484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:57.664087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:57.950991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:57.959894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-58657138-48f1-4b44-9f9f-80db4fa75faf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.331037 Sep 18 06:26:57.962945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 185/377] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:57.969963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:57.970888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:57.977055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:57.977564 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:57.978002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:57.978807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:57.979252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:57.979701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:58.156233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3317aa8a-0d5b-4a0f-980a-23302bd8823b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/action" status: 202 len: 0 microversion: 2.1 time: 0.863837 Sep 18 06:26:58.156871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 193/378] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:26:57 2022] POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/action => generated 0 bytes in 865 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:26:58.165628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:58.166236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:58.173754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:58.174522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:58.174988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:58.175690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:58.176134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:58.176551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:58.199253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:58.211697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ce318779-03f9-4a9c-a427-42900aec5ef0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.247266 Sep 18 06:26:58.212526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 186/379] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:57 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:58.217560 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-b357cd5c-6ed8-4570-bd4b-5d3b6cecd7e4 req-0e7edfd3-f5d4-4f48-a347-d555b50873cc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b4bf8fc2-b57b-4198-939c-bd88739c785e", "tag": "1956a12e-1f36-454a-baa5-f949db978b94"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:58.234465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-b357cd5c-6ed8-4570-bd4b-5d3b6cecd7e4 req-0e7edfd3-f5d4-4f48-a347-d555b50873cc service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:58.234979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-b357cd5c-6ed8-4570-bd4b-5d3b6cecd7e4 req-0e7edfd3-f5d4-4f48-a347-d555b50873cc service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:58.235411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-b357cd5c-6ed8-4570-bd4b-5d3b6cecd7e4 req-0e7edfd3-f5d4-4f48-a347-d555b50873cc service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:58.247880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-b357cd5c-6ed8-4570-bd4b-5d3b6cecd7e4 req-0e7edfd3-f5d4-4f48-a347-d555b50873cc service nova] Creating event network-changed:1956a12e-1f36-454a-baa5-f949db978b94 for instance b4bf8fc2-b57b-4198-939c-bd88739c785e on hv-cinder-85535 Sep 18 06:26:58.253050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-b357cd5c-6ed8-4570-bd4b-5d3b6cecd7e4 req-0e7edfd3-f5d4-4f48-a347-d555b50873cc service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039204 Sep 18 06:26:58.254014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 187/380] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:26:58.355493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:58.370439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:58.379306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-32feffc3-036c-42da-a00e-a47e18a3509d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1762 microversion: 2.1 time: 0.217406 Sep 18 06:26:58.391935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 194/381] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:58 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1762 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:58.649837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:58.650532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:58.656565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:58.657042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:58.657478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:58.658334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:58.658807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:58.659230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:58.800307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:58.808650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-86606e1a-1a99-4ffa-8678-d8de6f3f95ff tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1319 microversion: 2.1 time: 0.163986 Sep 18 06:26:58.809222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 188/382] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:58 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1319 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:58.983119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:58.983865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:58.992319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:58.992922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:58.993427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:58.994367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:58.994863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:58.995357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:59.118327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-738a51f8-9750-42d2-b43f-a9d7f4a5ea2d req-6b8df302-f9a1-4e67-8e6f-95239698c312 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a8254cee-e629-4d82-988c-708bac734c1e", "name": "network-vif-plugged", "status": "completed", "tag": "7ff7d3c9-95be-4eea-b0b3-a8bd5d358217"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:26:59.128313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-738a51f8-9750-42d2-b43f-a9d7f4a5ea2d req-6b8df302-f9a1-4e67-8e6f-95239698c312 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:59.129197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-738a51f8-9750-42d2-b43f-a9d7f4a5ea2d req-6b8df302-f9a1-4e67-8e6f-95239698c312 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:59.129844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-738a51f8-9750-42d2-b43f-a9d7f4a5ea2d req-6b8df302-f9a1-4e67-8e6f-95239698c312 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:59.139263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:59.148761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-738a51f8-9750-42d2-b43f-a9d7f4a5ea2d req-6b8df302-f9a1-4e67-8e6f-95239698c312 service nova] Creating event network-vif-plugged:7ff7d3c9-95be-4eea-b0b3-a8bd5d358217 for instance a8254cee-e629-4d82-988c-708bac734c1e on hv-cinder-85535 Sep 18 06:26:59.152909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-972c5813-2e73-43cd-bdab-3e7eb03e23f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.173710 Sep 18 06:26:59.153493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 195/383] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:58 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:59.168041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-738a51f8-9750-42d2-b43f-a9d7f4a5ea2d req-6b8df302-f9a1-4e67-8e6f-95239698c312 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039805 Sep 18 06:26:59.168584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 189/384] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:26:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:26:59.235517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:59.236143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:59.242239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:59.242840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:59.243345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:59.244105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:59.244637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:59.245147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:59.382870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:59.391181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a34acea3-67d1-4b4a-aa91-2b204ef9d775 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.160707 Sep 18 06:26:59.391757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 196/385] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:59 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:26:59.833109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:26:59.833815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:26:59.840053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:59.840534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:59.840968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:59.841712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:26:59.842232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:26:59.842695 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:26:59.981145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:26:59.988943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-edf5eba8-9bb8-4872-9328-a37cb97d158e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1508 microversion: 2.1 time: 0.160464 Sep 18 06:26:59.990142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 190/386] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:26:59 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1508 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:00.178232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:00.179102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:00.189033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:00.189616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:00.190183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:00.191197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:00.191823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:00.192298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:00.388436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:00.397072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f41c3f44-b0c8-4edf-a7e7-0647c1204929 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.225062 Sep 18 06:27:00.397724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 197/387] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:27:00.415926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:00.416606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:00.423387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:00.423943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:00.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:00.425262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:00.425712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:00.426134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:00.430528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:00.587725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:00.595782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0ce73e12-ae77-4167-9873-f7a5d75f2b09 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.183659 Sep 18 06:27:00.596345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 191/388] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:00 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:01.420377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:01.421016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:01.427406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:01.427868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:01.428306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:01.428991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:01.429451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:01.429874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:01.568255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:01.577003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b7c853f6-94fa-4c4a-900e-9afe02b5fae1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.160538 Sep 18 06:27:01.577722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 198/389] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:01.620045 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:01.621281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:01.628072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:01.628553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:01.628982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:01.629673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:01.630763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:01.631435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:01.763666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:01.771327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-be6bea0a-b68a-4c05-b7c2-f1ad9052cc23 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.155184 Sep 18 06:27:01.771886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 192/390] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:01 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:02.601055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:02.602863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:02.611612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:02.612449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:02.612926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:02.613629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:02.614081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:02.614511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:02.717383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-276a0aa0-0edb-4e8d-9372-cfeaf7324d4b req-00153501-ace5-4e34-87ad-e62f058173c7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2131eb93-ac09-4fc4-991e-3a0cc04b5bbd", "name": "network-vif-plugged", "status": "completed", "tag": "96889ca2-75ee-41de-b4a7-a0b675cf59ff"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:02.725281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=24 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:02.733521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-276a0aa0-0edb-4e8d-9372-cfeaf7324d4b req-00153501-ace5-4e34-87ad-e62f058173c7 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:02.734258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-276a0aa0-0edb-4e8d-9372-cfeaf7324d4b req-00153501-ace5-4e34-87ad-e62f058173c7 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:02.734716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-276a0aa0-0edb-4e8d-9372-cfeaf7324d4b req-00153501-ace5-4e34-87ad-e62f058173c7 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:02.747719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-276a0aa0-0edb-4e8d-9372-cfeaf7324d4b req-00153501-ace5-4e34-87ad-e62f058173c7 service nova] Creating event network-vif-plugged:96889ca2-75ee-41de-b4a7-a0b675cf59ff for instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd on hv-cinder-85535 Sep 18 06:27:02.752678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-276a0aa0-0edb-4e8d-9372-cfeaf7324d4b req-00153501-ace5-4e34-87ad-e62f058173c7 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039151 Sep 18 06:27:02.753499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 193/391] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:27:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:02.774795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:02.793654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-00910026-1c8c-4a6c-9729-a90939e0f5d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1371 microversion: 2.1 time: 0.196840 Sep 18 06:27:02.794447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 199/392] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1371 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:02.796681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:02.797658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:02.807071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:02.807565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:02.808249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:02.808974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:02.809456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:02.809936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:02.969535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:02.977605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6d5c83c1-c729-430d-9bb2-b41abf9b951f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.186558 Sep 18 06:27:02.978182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 194/393] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:02 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:27:03.824370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:03.825017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:03.831721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:03.832212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:03.832652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:03.833405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:03.833862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:03.834293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:03.968896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:03.978220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dc0c1423-54e7-49bd-aa6a-55a49f05c44d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1564 microversion: 2.1 time: 0.157790 Sep 18 06:27:03.979017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 200/394] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1564 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:04.000872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:04.001377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:04.008063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:04.008203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:04.008653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:04.009093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:04.009230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:04.009782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:04.010233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:04.010699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:04.015788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:04.016275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:04.016743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:04.017486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:04.017947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:04.018377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:04.120220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2e80e4e9-d4b7-4b38-a8ab-4601f88bf369 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.123301 Sep 18 06:27:04.120818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 195/395] 10.222.0.20 () {58 vars in 1275 bytes} [Sun Sep 18 06:27:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-interface => generated 284 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:04.184943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:04.193362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2973321c-13c9-4687-8054-7e819ba3d879 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.189514 Sep 18 06:27:04.193931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 201/396] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:04 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:05.216464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:05.217286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:05.223853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:05.224339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:05.224773 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:05.225452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:05.225900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:05.226321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:05.312511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-039de4cb-e167-42ea-a77f-b26cf28cbb69 req-e097fe89-be2e-4a33-b388-23526823854c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a8254cee-e629-4d82-988c-708bac734c1e", "tag": "7ff7d3c9-95be-4eea-b0b3-a8bd5d358217"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:05.322391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-039de4cb-e167-42ea-a77f-b26cf28cbb69 req-e097fe89-be2e-4a33-b388-23526823854c service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:05.323093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-039de4cb-e167-42ea-a77f-b26cf28cbb69 req-e097fe89-be2e-4a33-b388-23526823854c service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:05.323522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-039de4cb-e167-42ea-a77f-b26cf28cbb69 req-e097fe89-be2e-4a33-b388-23526823854c service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:05.335969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-039de4cb-e167-42ea-a77f-b26cf28cbb69 req-e097fe89-be2e-4a33-b388-23526823854c service nova] Creating event network-changed:7ff7d3c9-95be-4eea-b0b3-a8bd5d358217 for instance a8254cee-e629-4d82-988c-708bac734c1e on hv-cinder-85535 Sep 18 06:27:05.342885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-039de4cb-e167-42ea-a77f-b26cf28cbb69 req-e097fe89-be2e-4a33-b388-23526823854c service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033973 Sep 18 06:27:05.343598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 202/397] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:27:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:27:05.386451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:05.394398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-93774ded-7724-49a8-98d5-12da654e873f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1367 microversion: 2.1 time: 0.182414 Sep 18 06:27:05.395758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 196/398] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:05 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1367 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:06.420370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:06.421069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:06.427771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:06.428252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:06.428683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:06.429365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:06.429831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:06.430236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:06.565442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:06.573245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5c2ac60e-85c4-4f1d-bfe1-e1d572112bd9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1558 microversion: 2.1 time: 0.157678 Sep 18 06:27:06.573823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 203/399] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:06 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:06.595571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:06.596259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:06.603485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:06.603958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:06.604390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:06.605061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:06.605506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:06.605951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:06.689132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0fc9808f-ea6a-4f1e-9215-b58bcb97497e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.097843 Sep 18 06:27:06.689784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 197/400] 10.222.0.20 () {58 vars in 1275 bytes} [Sun Sep 18 06:27:06 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd/os-interface => generated 285 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:07.654524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-fe0c2155-84e3-4b2f-8d28-6639cb7d0547 req-22a546de-b159-4ef7-afb6-7eca6e47aa55 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2131eb93-ac09-4fc4-991e-3a0cc04b5bbd", "tag": "96889ca2-75ee-41de-b4a7-a0b675cf59ff"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:07.665423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fe0c2155-84e3-4b2f-8d28-6639cb7d0547 req-22a546de-b159-4ef7-afb6-7eca6e47aa55 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:07.665903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fe0c2155-84e3-4b2f-8d28-6639cb7d0547 req-22a546de-b159-4ef7-afb6-7eca6e47aa55 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:07.666909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fe0c2155-84e3-4b2f-8d28-6639cb7d0547 req-22a546de-b159-4ef7-afb6-7eca6e47aa55 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:07.679549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-fe0c2155-84e3-4b2f-8d28-6639cb7d0547 req-22a546de-b159-4ef7-afb6-7eca6e47aa55 service nova] Creating event network-changed:96889ca2-75ee-41de-b4a7-a0b675cf59ff for instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd on hv-cinder-85535 Sep 18 06:27:07.684652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-fe0c2155-84e3-4b2f-8d28-6639cb7d0547 req-22a546de-b159-4ef7-afb6-7eca6e47aa55 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033823 Sep 18 06:27:07.685611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 204/401] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:27:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:17.770252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:17.771170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=32 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:17.775858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1244079497", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1890991792", "security_groups": [{"name": "tempest-secgroup-smoke-749347155"}], "networks": [{"uuid": "a38636c7-0454-4559-9412-509e694410a8"}], "block_device_mapping_v2": [{"uuid": "684d257f-1b1f-4494-991a-0dde53cbae6e", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:17.898402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] validate_networks() for [('a38636c7-0454-4559-9412-509e694410a8', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:27:18.273104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:27:18.273529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:27:18.274036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:27:18.274448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:27:18.274900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:27:18.283928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:18.284410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:18.284843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:18.286158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:18.286648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:18.287080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:18.305165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Getting quotas for project d0adbb8b987449f08ee39d325424b1da. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:27:18.309737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Getting quotas for user c2b511ca42e6427aa2f1627c87ad3fad and project d0adbb8b987449f08ee39d325424b1da. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:27:18.372351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:27:18.374869 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='684d257f-1b1f-4494-991a-0dde53cbae6e',volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:27:18.523261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-248b6465-e48e-4ba4-a07b-5afdae05c3ff tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.754344 Sep 18 06:27:18.524254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 198/402] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:27:17 2022] POST /compute/v2.1/servers => generated 405 bytes in 756 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:27:18.536279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d310c4f9-839c-4b28-9dc0-5c100d6782ea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:18.536996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d310c4f9-839c-4b28-9dc0-5c100d6782ea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:18.539980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:18.540732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=52 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:18.604042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d310c4f9-839c-4b28-9dc0-5c100d6782ea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:18.610929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d310c4f9-839c-4b28-9dc0-5c100d6782ea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1089 microversion: 2.1 time: 0.079087 Sep 18 06:27:18.611513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 205/403] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:18 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1089 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:19.635269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:19.635850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:19.642764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:19.643261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:19.643723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:19.644950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:19.645430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:19.645855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:19.709012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:19.716806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3b94ba01-8813-4dd7-a926-c8b3006875af tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1187 microversion: 2.1 time: 0.086456 Sep 18 06:27:19.717373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 199/404] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:19 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1187 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:20.741919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:20.743326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:20.750059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:20.750888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:20.751491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:20.752441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:20.753069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:20.753627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:20.907511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:20.921775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b2294e99-295d-4c0f-93ab-66198bc32f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1272 microversion: 2.1 time: 0.185012 Sep 18 06:27:20.923362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 206/405] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:20 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1272 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:21.442239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-43cde3a2-6f82-4e5d-8d9f-c5ae3d0822df req-c94a5808-d5f9-418e-a48e-7e5c2e75a593 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ce28d02-87b5-461d-bac3-2bf15b26fea9", "tag": "726ec9db-ceda-4200-b057-55e60adeb8a9"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:21.454132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-43cde3a2-6f82-4e5d-8d9f-c5ae3d0822df req-c94a5808-d5f9-418e-a48e-7e5c2e75a593 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:21.454864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-43cde3a2-6f82-4e5d-8d9f-c5ae3d0822df req-c94a5808-d5f9-418e-a48e-7e5c2e75a593 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:21.455374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-43cde3a2-6f82-4e5d-8d9f-c5ae3d0822df req-c94a5808-d5f9-418e-a48e-7e5c2e75a593 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:21.469634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-43cde3a2-6f82-4e5d-8d9f-c5ae3d0822df req-c94a5808-d5f9-418e-a48e-7e5c2e75a593 service nova] Creating event network-changed:726ec9db-ceda-4200-b057-55e60adeb8a9 for instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 on hv-cinder-85535 Sep 18 06:27:21.475302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-43cde3a2-6f82-4e5d-8d9f-c5ae3d0822df req-c94a5808-d5f9-418e-a48e-7e5c2e75a593 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037886 Sep 18 06:27:21.476239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 200/406] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:27:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:21.946903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:21.947433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:21.954275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:21.954815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:21.955278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:21.955979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:21.956432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:21.957373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:22.089530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:22.097312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e23187d-e978-4f6d-8576-93288df485f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.154525 Sep 18 06:27:22.098462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 207/407] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:21 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:23.121731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:23.122474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:23.128771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:23.129259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:23.129726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:23.130440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:23.131182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:23.131606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:23.262823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:23.271141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a50a7485-d93f-4825-baa3-27b3c38831aa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.154072 Sep 18 06:27:23.271701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 201/408] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:23 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:24.293568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:24.294180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:24.301161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:24.301641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:24.302076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:24.302793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:24.303261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:24.303685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:24.431203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:24.438836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-67049839-6e23-46e4-93c6-baf1057ad2c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.149081 Sep 18 06:27:24.439404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 208/409] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:24 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:24.851721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "360e2653-c270-4d5c-9445-519b9d3ac836", "device": "/dev/sdb"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:24.854534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:24.860982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:24.861468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:24.861904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:24.862613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:24.863088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:24.863513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:25.465156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:25.465898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:25.473142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:25.473672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:25.474150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:25.474900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:25.475359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:25.475968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:25.481569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a07f899d-fb9f-403d-948b-c4ac7f653aab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "POST /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.634288 Sep 18 06:27:25.482208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 202/410] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:27:24 2022] POST /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e/os-volume_attachments => generated 194 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:25.640119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:25.648418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fa1ce9e6-3a9b-4b91-8dc4-183d76f676fe tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.187751 Sep 18 06:27:25.649081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 209/411] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:25 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:26.671928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:26.672626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:26.679723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:26.680212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:26.680653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:26.681352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:26.681835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:26.682262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:26.813944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:26.821671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e5326853-e927-44f0-87c5-fb032c2e0880 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.154465 Sep 18 06:27:26.822236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 203/412] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:27.844109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:27.844641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:27.851947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:27.852430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:27.852866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:27.853552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:27.854026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:27.854449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:27.980818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:27.988840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-929472b6-1b98-42ce-9f19-f6348746d453 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.148668 Sep 18 06:27:27.989397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 210/413] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:27 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:29.012098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:29.012720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:29.015329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:29.016123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:29.022750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:29.023249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:29.023690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:29.024391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:29.024996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:29.025429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:29.157054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:29.164872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-62b0131f-0656-4850-8c4d-efe8f0aa1f08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.156608 Sep 18 06:27:29.165444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 204/414] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:29 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:30.187080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:30.187710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:30.189574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:30.195890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:30.196365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:30.196796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:30.197477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:30.197927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:30.198349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:30.324027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:30.332211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2ba12e47-2e1e-498d-80a5-6d58346ae3f1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.148948 Sep 18 06:27:30.332766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 211/415] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:30 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:31.354680 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:31.355318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:31.361659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:31.362138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:31.362750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:31.363432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:31.363877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:31.364300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:31.500708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:31.508966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a5bc9111-01ad-4893-bb0d-427f40125ab4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.158151 Sep 18 06:27:31.509540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 205/416] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:31 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:32.532133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:32.532746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:32.539233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:32.539717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:32.540152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:32.540836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:32.541288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:32.541718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:32.668406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:32.677015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d1e4ff90-1b6b-4807-a2d7-f6340d1a3655 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.148780 Sep 18 06:27:32.677598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 212/417] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:32 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:33.058101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-ec19f816-b84f-495f-9edc-2f0453c4e0a4 req-c97ae96d-e379-4cc0-a869-6cf239c5cb49 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ce28d02-87b5-461d-bac3-2bf15b26fea9", "name": "network-vif-plugged", "status": "completed", "tag": "726ec9db-ceda-4200-b057-55e60adeb8a9"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:33.068563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ec19f816-b84f-495f-9edc-2f0453c4e0a4 req-c97ae96d-e379-4cc0-a869-6cf239c5cb49 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:33.069034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ec19f816-b84f-495f-9edc-2f0453c4e0a4 req-c97ae96d-e379-4cc0-a869-6cf239c5cb49 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:33.069539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ec19f816-b84f-495f-9edc-2f0453c4e0a4 req-c97ae96d-e379-4cc0-a869-6cf239c5cb49 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:33.081704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-ec19f816-b84f-495f-9edc-2f0453c4e0a4 req-c97ae96d-e379-4cc0-a869-6cf239c5cb49 service nova] Creating event network-vif-plugged:726ec9db-ceda-4200-b057-55e60adeb8a9 for instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 on hv-cinder-85535 Sep 18 06:27:33.086959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-ec19f816-b84f-495f-9edc-2f0453c4e0a4 req-c97ae96d-e379-4cc0-a869-6cf239c5cb49 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032692 Sep 18 06:27:33.087930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 206/418] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:27:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:33.699471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:33.700010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:33.706341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:33.706851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:33.707286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:33.707968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:33.708418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:33.708842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:33.835585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:33.843140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-943762a4-366b-4af8-be88-5dd8e0869f94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.147585 Sep 18 06:27:33.843703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 213/419] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:33 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:34.867744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:34.868460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:34.875032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:34.875508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:34.875937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:34.876614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:34.877060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:34.877478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:35.250404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:35.258362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-673af3a0-aef4-4adc-b98e-b0f2fa409967 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.395194 Sep 18 06:27:35.258954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 207/420] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:34 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:36.281581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:36.282198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:36.289200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:36.289686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:36.290126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:36.290844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:36.291297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:36.291722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:36.417715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:36.425351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eba1e975-d885-4de9-b3e8-09485b371736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.147633 Sep 18 06:27:36.425921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 214/421] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:36 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:37.447834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:37.448550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:37.455536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:37.456100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:37.456571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:37.457286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:37.457731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:37.458157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:37.598334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:37.606122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-941bc500-4c5d-47ea-8c93-999f43effb66 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.162389 Sep 18 06:27:37.606710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 208/422] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:37 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:38.629107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:38.629673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:38.636284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:38.636779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:38.637195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:38.637874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:38.638327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:38.638787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:38.773294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:38.780907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cc7432f6-333c-493e-a9fc-3e694dd91c34 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.156386 Sep 18 06:27:38.781469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 215/423] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:38 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:39.484555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:39.485051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:39.491650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:39.492128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:39.492561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:39.493243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:39.493695 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:39.494123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:39.517284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b0c25cc4-c382-4158-9c35-db3a9eac66e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/ips" status: 200 len: 151 microversion: 2.20 time: 0.036679 Sep 18 06:27:39.517834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 209/424] 10.222.0.20 () {60 vars in 1289 bytes} [Sun Sep 18 06:27:39 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/ips => generated 151 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 18 06:27:39.804505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:39.805309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:39.812319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:39.812875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:39.813385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:39.814146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:39.814844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:39.815975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:39.951390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:39.959077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eb2be931-741b-4f58-af91-f11fc1b35524 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.158629 Sep 18 06:27:39.959636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 216/425] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:39 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:40.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:40.981778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:40.988120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:40.988602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:40.989031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:40.989714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:40.990165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:40.990600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:41.118024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:41.125643 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-05819a2c-2aca-4099-841f-9c95fb59abc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.148334 Sep 18 06:27:41.126202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 210/426] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:40 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:42.149460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:42.150095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:42.156812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:42.157291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:42.157723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:42.158414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:42.158913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:42.159333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:42.295923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:42.304106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-51237319-cab1-4cc8-82eb-020c6c629e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1260 microversion: 2.1 time: 0.158601 Sep 18 06:27:42.304669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 217/427] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:42 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:43.328024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:43.328644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:43.335118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:43.335606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:43.336045 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:43.336742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:43.337197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:43.337629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:43.468946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:43.477166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-310e0677-9e0e-4a02-8072-2d4749d5fca7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1445 microversion: 2.1 time: 0.153084 Sep 18 06:27:43.477731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 211/428] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:43 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1445 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:43.500061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:43.500668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:43.507226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:43.507700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:43.508129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:43.508807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:43.509253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:43.509671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:43.638941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:43.647923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-07dbf292-5833-49f8-b7cd-cd6aa4b41858 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1445 microversion: 2.1 time: 0.151613 Sep 18 06:27:43.648486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 218/429] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:43 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1445 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:44.058547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:44.059136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:44.066459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:44.066983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:44.067423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:44.068136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:44.068640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:44.069094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:44.092442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lazy-loading 'flavor' on Instance uuid 5ed3cfa3-6093-4c26-9877-01470acd2e16 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:27:44.292603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ab56a56b-6d02-4500-bdf9-ca358da5b385 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/action" status: 202 len: 0 microversion: 2.20 time: 0.237797 Sep 18 06:27:44.293542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 212/430] 10.222.0.20 () {62 vars in 1319 bytes} [Sun Sep 18 06:27:44 2022] POST /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/action => generated 0 bytes in 240 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:27:44.303534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:44.304149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:44.310646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:44.311070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:44.311542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:44.312262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:44.312993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:44.313574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:44.551720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:44.559953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:44.561488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:44.562733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ed0c09ab-66e8-4764-a785-cf20026305f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.264799 Sep 18 06:27:44.563304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 219/431] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:44 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:45.356625 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-713f522d-ae4b-4466-9b9e-e1bda05bb29d req-0486abbc-1cbb-4121-8346-649e84d3f1e3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ce28d02-87b5-461d-bac3-2bf15b26fea9", "tag": "726ec9db-ceda-4200-b057-55e60adeb8a9"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:45.366753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-713f522d-ae4b-4466-9b9e-e1bda05bb29d req-0486abbc-1cbb-4121-8346-649e84d3f1e3 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:45.367303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-713f522d-ae4b-4466-9b9e-e1bda05bb29d req-0486abbc-1cbb-4121-8346-649e84d3f1e3 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:45.367811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-713f522d-ae4b-4466-9b9e-e1bda05bb29d req-0486abbc-1cbb-4121-8346-649e84d3f1e3 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:45.396815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-713f522d-ae4b-4466-9b9e-e1bda05bb29d req-0486abbc-1cbb-4121-8346-649e84d3f1e3 service nova] Creating event network-changed:726ec9db-ceda-4200-b057-55e60adeb8a9 for instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 on hv-cinder-85535 Sep 18 06:27:45.403970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-713f522d-ae4b-4466-9b9e-e1bda05bb29d req-0486abbc-1cbb-4121-8346-649e84d3f1e3 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051899 Sep 18 06:27:45.404807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 213/432] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:27:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:45.588171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:45.588894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:45.595574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:45.596494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:45.596934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:45.597634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:45.598091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:45.598521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:45.727797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:45.736195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:45.738334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:45.739524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-71cb52fd-67dd-4c2c-92fa-fff5c6a07b1d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.155388 Sep 18 06:27:45.740084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 220/433] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:45 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:46.765707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:46.766395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:46.772530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:46.773004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:46.773437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:46.774109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:46.774556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:46.775006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:46.778949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:46.907296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:46.916193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:46.917573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:46.918758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-eb836d02-7849-4d10-8073-30a7bc90b676 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.157848 Sep 18 06:27:46.919301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 214/434] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:46 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:47.945138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:47.945742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:47.952419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:47.952897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:47.953340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:47.954003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:47.954448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:47.954895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:48.081743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:48.089740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:48.091234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:48.092416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4ce41fa7-87ce-449f-84b2-6285ab9248da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.151810 Sep 18 06:27:48.093282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 221/435] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:47 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:48.555060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:48.557201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:48.559499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=48,nova_api:INSERT=3 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:48.564675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:48.565152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:48.565586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:48.566264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:48.566750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:48.567173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:48.636647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6a91a98e-6ea0-44cf-bd78-89f7f05086c0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/action" status: 202 len: 0 microversion: 2.1 time: 0.085864 Sep 18 06:27:48.637167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 215/436] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:27:48 2022] POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/action => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:27:48.645422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:48.646129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:48.648296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=37 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:48.653650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:48.654298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:48.654904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:48.655649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:48.656130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:48.656564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:48.659577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=97 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:48.802465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:48.810742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1624cb2f-dab5-4ff4-835d-b9c7a3dd14ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1754 microversion: 2.1 time: 0.169267 Sep 18 06:27:48.811305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 222/437] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:48 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1754 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:49.118284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:49.119021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:49.125449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:49.125937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:49.126367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:49.127100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:49.127548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:49.127969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:49.274606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:49.283679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:49.285120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:49.286274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b0e0e944-f9ea-43aa-a783-767ef821f6b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.172754 Sep 18 06:27:49.286866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 216/438] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:49 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:49.835118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:49.835656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:49.841975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:49.842456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:49.843039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:49.843753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:49.844206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:49.844627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:49.982273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:49.990393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9f60661b-2313-4ae6-9256-7334fcf2f964 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.159830 Sep 18 06:27:49.990985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 223/439] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:49 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:50.274328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:50.274844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:50.281229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:50.281707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:50.282136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:50.282834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:50.283282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:50.283697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:50.311312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:50.311919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:50.318408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:50.318916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:50.319346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:50.320024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:50.320471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:50.320890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:50.414016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid 141e3677-77e2-4021-918a-5fbe5292398e {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:27:50.456983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:50.465242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:50.466717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:50.467878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5eedf1ab-316d-4faf-9b8d-0a98ab244734 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.160456 Sep 18 06:27:50.468432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 224/440] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:50 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:27:50.525245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ffa318dc-a124-4a6c-bfef-89c2b4b37062 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e/os-volume_attachments/360e2653-c270-4d5c-9445-519b9d3ac836" status: 202 len: 0 microversion: 2.1 time: 0.254560 Sep 18 06:27:50.525823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 217/441] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:27:50 2022] DELETE /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e/os-volume_attachments/360e2653-c270-4d5c-9445-519b9d3ac836 => generated 0 bytes in 256 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:27:51.014657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:51.015262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:51.021978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:51.022456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:51.023032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:51.023711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:51.024162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:51.024584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:51.167636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:51.175764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c02a6228-991d-4a9f-8d18-a9d105aa8602 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.165137 Sep 18 06:27:51.176329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 225/442] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:51 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:51.493574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:51.494263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:51.500451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:51.500928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:51.501363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:51.502041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:51.502486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:51.502933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:51.637798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:51.648085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:51.649688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:51.651559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fdbcc791-9750-4b09-973a-39671eb7927a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.162762 Sep 18 06:27:51.652196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 218/443] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:51 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:52.198348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:52.199003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:52.206679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:52.207342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:52.208632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:52.209324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:52.209776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:52.210198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:52.351033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:52.359449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b21f7d3f-0c6d-46db-a68b-28cece9d216f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.164905 Sep 18 06:27:52.360022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 226/444] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:52 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:52.678352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:52.679026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:52.685266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:52.685745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:52.686175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:52.686880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:52.687331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:52.687750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:52.729797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:52.730211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:52.736120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:52.736600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:52.737029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:52.737713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:52.738161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:52.738596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:52.764507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:27:52.814236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:52.823851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:52.825263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:52.826411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-75ba9e81-0b7f-4847-8b74-4326ec8332b9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.152480 Sep 18 06:27:52.827105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 219/445] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:52 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:52.841162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e7071ae5-5ed9-4dcc-af48-f926926e0c8a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 204 len: 0 microversion: 2.1 time: 0.115015 Sep 18 06:27:52.841972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 227/446] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:27:52 2022] DELETE /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 18 06:27:52.851094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:52.852000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:52.857789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:52.858274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:52.858813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:52.859502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:52.859951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:52.860371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:52.987968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:52.996210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ec44bbeb-be03-4d66-89f7-6859b19e5871 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1514 microversion: 2.1 time: 0.150072 Sep 18 06:27:52.996767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 220/447] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:52 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1514 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:53.383570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:53.384198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:53.390625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:53.391116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:53.391553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:53.392271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:53.392727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:53.393153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:53.534553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:53.542740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a8afdf8b-b1c7-4b03-a718-7e89e4efd6a8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.164565 Sep 18 06:27:53.543309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 228/448] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:53 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:53.853901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:53.854501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:53.860883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:53.861361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:53.861818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:53.862497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:53.862968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:53.863389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:53.991205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:53.999181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:54.000622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:54.001760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b14dc0b0-2385-42fb-b751-63bb9157c885 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.151808 Sep 18 06:27:54.002316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 221/449] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:53 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:54.019487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:54.020607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:54.026536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:54.027050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:54.027486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:54.028169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:54.028627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:54.029050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:54.155641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:54.163970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-870a88be-d150-4634-95c1-cf3854b18a5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1514 microversion: 2.1 time: 0.148284 Sep 18 06:27:54.164534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 229/450] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:54 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1514 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:54.566346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:54.567427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:54.573372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:54.573856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:54.574298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:54.574998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:54.575452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:54.575877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:54.717995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:54.726998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a95a2ba0-6ce7-487c-ad0e-3d3c521ca2ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.164546 Sep 18 06:27:54.727558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 222/451] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:54 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:55.027136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:55.027742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:55.034096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:55.034593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:55.035040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:55.036406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:55.036859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:55.037280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:55.170105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:55.178335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:55.179821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:55.181550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-67af9b59-f20e-438e-9974-6190cf339d9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.157865 Sep 18 06:27:55.186937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:55.187623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:55.194940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:55.195049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 230/452] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:55 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:55.195507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:55.196063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:55.196820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:55.197336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:55.197822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:55.262240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:55.275258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-888cfe84-fa02-4887-ab07-feec756cd19c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1472 microversion: 2.1 time: 0.091975 Sep 18 06:27:55.275842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 223/453] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:55 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1472 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:55.298283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-3c42d62f-1e71-420e-9ba2-2a234f9da416 req-eb9bcdb3-fa58-45ec-a779-c0ea447dd9ca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "141e3677-77e2-4021-918a-5fbe5292398e", "name": "network-vif-deleted", "tag": "5a5c550c-fee5-4d01-961b-e15307eef48d"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:27:55.310681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-3c42d62f-1e71-420e-9ba2-2a234f9da416 req-eb9bcdb3-fa58-45ec-a779-c0ea447dd9ca service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:55.311159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-3c42d62f-1e71-420e-9ba2-2a234f9da416 req-eb9bcdb3-fa58-45ec-a779-c0ea447dd9ca service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:55.311582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-3c42d62f-1e71-420e-9ba2-2a234f9da416 req-eb9bcdb3-fa58-45ec-a779-c0ea447dd9ca service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:55.325454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-3c42d62f-1e71-420e-9ba2-2a234f9da416 req-eb9bcdb3-fa58-45ec-a779-c0ea447dd9ca service nova] Creating event network-vif-deleted:5a5c550c-fee5-4d01-961b-e15307eef48d for instance 141e3677-77e2-4021-918a-5fbe5292398e on hv-cinder-85535 Sep 18 06:27:55.330151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-3c42d62f-1e71-420e-9ba2-2a234f9da416 req-eb9bcdb3-fa58-45ec-a779-c0ea447dd9ca service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035678 Sep 18 06:27:55.330773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 231/454] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:27:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:27:55.748999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:55.749638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:55.756989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:55.757691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:55.758246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:55.759017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:55.759472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:55.760310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:55.906526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:55.915199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8f746f15-9598-404a-9624-3df8b348a777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.169967 Sep 18 06:27:55.915753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 224/455] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:55 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:56.207419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:56.208625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:56.215149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:56.215635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:56.216056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:56.216736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:56.217185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:56.217605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:56.298872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:56.299481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:56.306812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:56.307597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:56.308159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:56.308913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:56.309365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:56.309785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:56.346872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:56.347355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:56.347787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:56.348232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'fault' on Instance uuid 141e3677-77e2-4021-918a-5fbe5292398e {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:27:56.354214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:56.376587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:56.378046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:56.379268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2dea509e-c814-4b3d-9075-dd8177d21f42 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.176601 Sep 18 06:27:56.380686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 232/456] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:56 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:56.393624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:56.402621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ba4167db-ad4e-4235-a795-059e7194bc81 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 200 len: 1309 microversion: 2.1 time: 0.107886 Sep 18 06:27:56.403189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 225/457] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:56 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 1309 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:27:56.939073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:56.939680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:56.946179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:56.946683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:56.947114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:56.947790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:56.948237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:56.948656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:57.089273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:57.097988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bb06cef2-c633-4573-9d2c-ff36d243dc70 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.162547 Sep 18 06:27:57.099373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 233/458] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:56 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:57.406266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:57.406957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:57.413597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:57.414075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:57.414506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:57.415222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:57.415668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:57.416284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:57.428265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:57.428888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: 141e3677-77e2-4021-918a-5fbe5292398e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:57.437960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:57.438730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:57.439401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:57.440512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:57.441254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:57.441922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:57.464609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Instance 141e3677-77e2-4021-918a-5fbe5292398e could not be found. Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e HTTP/1.1 Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:27:57.467325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 141e3677-77e2-4021-918a-5fbe5292398e could not be found. Sep 18 06:27:57.468849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '141e3677-77e2-4021-918a-5fbe5292398e'} Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 141e3677-77e2-4021-918a-5fbe5292398e could not be found. Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:27:57.469743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 141e3677-77e2-4021-918a-5fbe5292398e could not be found. Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 141e3677-77e2-4021-918a-5fbe5292398e could not be found. Sep 18 06:27:57.471104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:27:57.473105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 404 to user: Instance 141e3677-77e2-4021-918a-5fbe5292398e could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:27:57.473105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8c11323b-3804-4ea4-8296-603332fb144d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e" status: 404 len: 111 microversion: 2.1 time: 0.046488 Sep 18 06:27:57.473105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 234/459] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:57 2022] GET /compute/v2.1/servers/141e3677-77e2-4021-918a-5fbe5292398e => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:27:57.565550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:57.574842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:57.576597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:57.577775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2bc70869-0cea-435b-ae98-8a3f9b6f0855 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.175573 Sep 18 06:27:57.578338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 226/460] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:57 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:58.180494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:58.181096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:58.187071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:58.187546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:58.187976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:58.188651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:58.189103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:58.189522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:58.326980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:58.335784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-facd903f-b2c8-4d23-b87f-a021ba163c12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.159189 Sep 18 06:27:58.336347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 235/461] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:58 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:58.603199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:58.603811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:58.610339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:58.610854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:58.611300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:58.611989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:58.612439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:58.612867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:58.743220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:58.751274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:58.752719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:58.753874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-40f800ae-d017-44c6-a455-d0721c42f36f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.154588 Sep 18 06:27:58.754437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 227/462] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:58 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:27:59.359099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:59.359709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:59.366853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:59.367332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:59.367766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:59.368447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:59.368897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:59.369320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:59.509908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:59.518208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5fc634e1-212a-46d7-9125-a82ef7e93772 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.163090 Sep 18 06:27:59.518802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 236/463] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:27:59 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:27:59.780053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:27:59.780669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:27:59.787645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:59.788143 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:59.788579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:59.789265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:27:59.789718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:27:59.790144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:27:59.918758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:59.922176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:59.928338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:59.929861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:27:59.931037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7fe914ae-75ca-407e-bfb7-e165a70706b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.154919 Sep 18 06:27:59.931624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 228/464] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:27:59 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:00.541168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:00.541781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:00.548208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:00.548690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:00.549123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:00.549811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:00.550264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:00.550720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:00.688231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:00.697164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1d992c18-7dfc-42d3-8d3c-2fb7c16416b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.159815 Sep 18 06:28:00.697735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 237/465] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:00 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:00.956283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:00.956891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:00.964592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:00.965077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:00.965512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:00.966196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:00.966671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:00.967120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:01.100447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:01.108485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:01.109934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:01.111131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6b7f92a3-a600-486a-8a9e-c2bd0f8b1f07 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1723 microversion: 2.20 time: 0.158741 Sep 18 06:28:01.111684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 229/466] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:00 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:01.720113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:01.720644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:01.727538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:01.728013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:01.728439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:01.729123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:01.729569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:01.729986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:01.744904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:01.884522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:01.893281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c51b79a8-cef6-46e3-8fed-ec959f395a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.177130 Sep 18 06:28:01.893841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 238/467] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:01 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:02.136773 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:02.137376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:02.143606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:02.144089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:02.144516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:02.145863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:02.146310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:02.146764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:02.277095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:02.285062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:02.286493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:02.287662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-acd40286-20f2-4cc7-99c6-d3cab25d003d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1744 microversion: 2.20 time: 0.154804 Sep 18 06:28:02.288240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 230/468] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:02 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1744 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:02.917550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:02.918166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:02.924755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:02.925237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:02.925700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:02.927007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:02.927460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:02.927885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:03.072400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:03.081291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a1ea5fd7-6c1f-44c7-9d38-415165dcd3a4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.167586 Sep 18 06:28:03.082190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 239/469] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:02 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:03.313486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:03.314128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:03.321071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:03.321554 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:03.321984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:03.322716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:03.323169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:03.323589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:03.450827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:03.459910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:03.461805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:03.463065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4d31f874-cd32-4991-9e3d-1542ff0ef4f7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1744 microversion: 2.20 time: 0.153398 Sep 18 06:28:03.463805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 231/470] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:03 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1744 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:04.105155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:04.105772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:04.112546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:04.113025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:04.113465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:04.114799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:04.115256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:04.115686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:04.265516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:04.273840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-77853ba9-adb5-4a83-acb7-95c8df18942a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.172506 Sep 18 06:28:04.274401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 240/471] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:04 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:04.488098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:04.488704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:04.495546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:04.496024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:04.496454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:04.497139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:04.497589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:04.498011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:04.626539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:04.634784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:04.636219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:04.637353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2669fda9-baa1-4a65-9479-a94d6a75a5b8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1744 microversion: 2.20 time: 0.153200 Sep 18 06:28:04.637893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 232/472] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:04 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1744 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:05.297159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:05.297800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:05.304641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:05.305116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:05.305544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:05.306225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:05.306704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:05.307126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:05.445752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:05.454055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-26eb975c-bc82-43e5-b901-b8f99c216c43 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.160744 Sep 18 06:28:05.454648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 241/473] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:05 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:05.663158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:05.663767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:05.670284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:05.670796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:05.671232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:05.671911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:05.672359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:05.672781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:05.799624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:05.808305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:05.809778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:05.811022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-24421fb6-a2e0-4fc1-a7d9-1fd8ed0149e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1744 microversion: 2.20 time: 0.152224 Sep 18 06:28:05.811562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 233/474] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:05 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1744 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:06.477426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:06.478043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:06.484522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:06.484997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:06.485430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:06.486121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:06.486567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:06.487028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:06.629875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:06.638218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-adb7cb1c-2035-4505-9b41-b978ad4f033d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.164768 Sep 18 06:28:06.638812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 242/475] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:06 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:06.754956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-4877a38b-efd9-44ae-8fdd-c826d967ba4f req-0de27510-b5b6-4baa-a959-b6b79728fcf0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ed3cfa3-6093-4c26-9877-01470acd2e16", "name": "network-vif-plugged", "status": "completed", "tag": "84893f57-6d56-4075-9e20-5bd22c657e44"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:06.766019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-4877a38b-efd9-44ae-8fdd-c826d967ba4f req-0de27510-b5b6-4baa-a959-b6b79728fcf0 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:06.766531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-4877a38b-efd9-44ae-8fdd-c826d967ba4f req-0de27510-b5b6-4baa-a959-b6b79728fcf0 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:06.766985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-4877a38b-efd9-44ae-8fdd-c826d967ba4f req-0de27510-b5b6-4baa-a959-b6b79728fcf0 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:06.777975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-4877a38b-efd9-44ae-8fdd-c826d967ba4f req-0de27510-b5b6-4baa-a959-b6b79728fcf0 service nova] Creating event network-vif-plugged:84893f57-6d56-4075-9e20-5bd22c657e44 for instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 on hv-cinder-85535 Sep 18 06:28:06.783291 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-4877a38b-efd9-44ae-8fdd-c826d967ba4f req-0de27510-b5b6-4baa-a959-b6b79728fcf0 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032943 Sep 18 06:28:06.784031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 234/476] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:28:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:06.836367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:06.837011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:06.842917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:06.843400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:06.843838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:06.844520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:06.844974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:06.845400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:06.978336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:06.986500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:06.987972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:06.989629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dfb9a880-c2c5-44cb-b005-34838208e69c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1744 microversion: 2.20 time: 0.157213 Sep 18 06:28:06.990352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 243/477] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:06 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1744 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:07.395169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1496221911", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "networks": [{"uuid": "3fc1a2fb-9883-4c17-86ff-23fe397da81c"}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:07.450147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] validate_networks() for [('3fc1a2fb-9883-4c17-86ff-23fe397da81c', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:28:07.663499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:07.664120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:07.670864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:07.671669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:07.672110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:07.672812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:07.673434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:07.673859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:08.183942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:08.188009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:28:08.188545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:28:08.188929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:28:08.189339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:28:08.189738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:28:08.199792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:08.200276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:08.200514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e2146e9d-7d01-4f16-8ad1-1d65b4e16ac9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.542184 Sep 18 06:28:08.200706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:08.201406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 244/478] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:07 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:08.206772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:08.207830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:08.208013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:08.208709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:08.209271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:08.215149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:08.215953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:08.216549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:08.217429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:08.217881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:08.218304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:08.237195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Getting quotas for project 1d431bdc9a2244dcb1bb589dee792d02. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:28:08.239047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Getting quotas for user e702f6ad9e38492497e0ae33abb9aae7 and project 1d431bdc9a2244dcb1bb589dee792d02. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:28:08.250898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:28:08.253356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:28:08.312344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a4df12c1-4d86-4a85-bb66-5ca1e08ebd58 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.921644 Sep 18 06:28:08.320016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 235/479] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:28:07 2022] POST /compute/v2.1/servers => generated 380 bytes in 923 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:28:08.324688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:08.325358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:08.332892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:08.333291 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:08.333733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:08.334445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:08.335241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:08.335681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:08.439039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:08.451667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:08.454150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:08.456080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lazy-loading 'services' on Instance uuid 045c9108-b1cd-4720-94bc-7aa9159c26d4 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:28:08.456679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1c4a6b50-096e-464b-9db6-6403e50fc099 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.253275 Sep 18 06:28:08.457897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 245/480] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:08.474524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a10a7f63-5cd0-4406-98c6-14b4e9c9d98f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:08.475168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a10a7f63-5cd0-4406-98c6-14b4e9c9d98f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:08.544808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-94d68593-fcba-4dfb-a5ce-5d17dbb3f439 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/os-volume_attachments/b853ab37-b2a4-461c-92c4-5966e90df9ca" status: 202 len: 0 microversion: 2.1 time: 0.230170 Sep 18 06:28:08.545697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 236/481] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:28:08 2022] DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/os-volume_attachments/b853ab37-b2a4-461c-92c4-5966e90df9ca => generated 0 bytes in 232 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:28:08.576429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a10a7f63-5cd0-4406-98c6-14b4e9c9d98f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:08.586813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a10a7f63-5cd0-4406-98c6-14b4e9c9d98f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1219 microversion: 2.1 time: 0.115946 Sep 18 06:28:08.587687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 246/482] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:08 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1219 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:09.226104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:09.228067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:09.240739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:09.242767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:09.243210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:09.244412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:09.245024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:09.247282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:09.464931 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:09.474616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-85f75fc1-90d9-4d42-a81a-9e641395fe49 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.254320 Sep 18 06:28:09.475346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 237/483] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:09 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:09.481859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:09.483142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:09.490834 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:09.491700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:09.492320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:09.493373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:09.493987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:09.494593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:09.612373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:09.613196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:09.621233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:09.623799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:09.624529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:09.630334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:09.632057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:09.632613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:09.711746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:09.722787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:09.725801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:09.727959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c3c0bb7e-42db-460f-8905-f9ea1a4c3695 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.250380 Sep 18 06:28:09.728961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 247/484] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:09 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:09.749315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:09.758263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-57de9b45-6f9c-4fec-94a1-c3809b416859 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1271 microversion: 2.1 time: 0.150789 Sep 18 06:28:09.759558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 238/485] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:09 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1271 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:10.508088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:10.508790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:10.521559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:10.522811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:10.523338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:10.524395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:10.525303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:10.525810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:10.737828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:10.753885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:10.754785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:10.760730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8c0b1b23-9139-46d2-9f5d-c4ddab523f3a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.263296 Sep 18 06:28:10.763068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:10.763876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 248/486] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:10 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:10.783420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:10.784356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:10.787354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:10.788045 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:10.790421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:10.791530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:10.795270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:10.808344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:10.809144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:10.809804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:10.811735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:10.812420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:10.815615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:10.956560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:10.965495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5ba8f751-698c-43f5-b796-7929b51ee03f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1277 microversion: 2.1 time: 0.183928 Sep 18 06:28:10.970334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 249/487] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:10 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1277 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:10.980646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:10.985952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-29e03dab-91b2-4ce7-a25c-1ec7b07638b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:10.989733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:10.993450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:10.995384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cc1c2868-045b-4ca0-b5b3-0f404936aca4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.245875 Sep 18 06:28:10.996668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 239/488] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:11.010769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29e03dab-91b2-4ce7-a25c-1ec7b07638b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:11.011620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29e03dab-91b2-4ce7-a25c-1ec7b07638b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:11.015015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29e03dab-91b2-4ce7-a25c-1ec7b07638b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:11.045590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-29e03dab-91b2-4ce7-a25c-1ec7b07638b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Going to try to stop instance {{(pid=120381) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 18 06:28:11.104720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-29e03dab-91b2-4ce7-a25c-1ec7b07638b1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/action" status: 202 len: 0 microversion: 2.1 time: 0.133080 Sep 18 06:28:11.105357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 250/489] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:28:10 2022] POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/action => generated 0 bytes in 134 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:28:11.116271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:11.117224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:11.125770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:11.126608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:11.127477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:11.130480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:11.131191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:11.131845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:11.321544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:11.332814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cc186078-2696-4fea-8b55-c85a635ac352 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.221417 Sep 18 06:28:11.335600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 240/490] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:11 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:11.788307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:11.789811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:11.797266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:11.798108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:11.798792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:11.799724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:11.800407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:11.801086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:11.998455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:11.998736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:12.001505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:12.010285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-04e1ff91-cd01-4428-be1d-d553b7db6682 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.226347 Sep 18 06:28:12.012053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 251/491] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:12.017252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:12.018027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:12.018534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:12.019390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:12.019922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:12.020409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:12.026412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:12.027094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:12.033439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:12.033964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:12.034638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:12.035558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:12.036332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:12.036955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:12.275962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:12.295926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-de502de7-40c7-48c9-a86c-1a0d07656692 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1319 microversion: 2.1 time: 0.302131 Sep 18 06:28:12.298289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:12.299886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 241/492] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:11 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1319 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:12.311308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:12.313227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:12.313674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-d6fb8e10-bdcf-471f-876e-5919da376cee req-72c8a640-7aab-4c66-b557-94b5bb653e66 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7", "tag": "3b32ba19-f05b-4744-adab-ddcb74d679c1"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:12.315171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-de1c15b2-4429-4701-8fa6-2c765e86193b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.294690 Sep 18 06:28:12.326640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 252/493] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:12 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:12.334929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d6fb8e10-bdcf-471f-876e-5919da376cee req-72c8a640-7aab-4c66-b557-94b5bb653e66 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:12.335416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d6fb8e10-bdcf-471f-876e-5919da376cee req-72c8a640-7aab-4c66-b557-94b5bb653e66 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:12.335849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d6fb8e10-bdcf-471f-876e-5919da376cee req-72c8a640-7aab-4c66-b557-94b5bb653e66 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:12.352849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-d6fb8e10-bdcf-471f-876e-5919da376cee req-72c8a640-7aab-4c66-b557-94b5bb653e66 service nova] Creating event network-changed:3b32ba19-f05b-4744-adab-ddcb74d679c1 for instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 on hv-cinder-85535 Sep 18 06:28:12.362314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:12.363601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-d6fb8e10-bdcf-471f-876e-5919da376cee req-72c8a640-7aab-4c66-b557-94b5bb653e66 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057452 Sep 18 06:28:12.363878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:12.364235 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 242/494] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:28:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:12.371417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:12.371960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:12.372567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:12.373428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:12.373987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:12.374417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:12.547760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:12.558250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ee0bf8c0-7006-4e73-8504-e2f328ea37fd tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.200984 Sep 18 06:28:12.574451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 253/495] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:12 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:13.042753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:13.044087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:13.055793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.056653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.057358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.058409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.059392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.060357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.243093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:13.254628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9bf0e9db-1975-409f-ad65-2eab7a1ba4f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.218682 Sep 18 06:28:13.255523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 243/496] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:13 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:13.323846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:13.325177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:13.333853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.334346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.334827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.335544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.335997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.336753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.342171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:13.344384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:13.357052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.357889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.358593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.359596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.360376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.361062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.569712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:13.583991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c5be2c9d-27cd-4e2d-a854-9f2fa076eee8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1319 microversion: 2.1 time: 0.263039 Sep 18 06:28:13.595577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:13.599999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 254/497] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:13 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1319 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:13.604216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:13.606644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:13.607861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-98c001d7-a77a-4205-86da-3830cc93409a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.269820 Sep 18 06:28:13.622615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 244/498] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:13 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:13.622839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6d3b844c-5d27-474e-a078-073931c04c5e"}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:13.622949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:13.622949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:13.623121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:13.623197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.623197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.623197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.623426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.623509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.623585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.623671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.623761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.624201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.624309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:13.624753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:13.625182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:13.630686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:13.774335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:13.786608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d8978d14-3ac3-4d9a-b36d-c5eb44a86a93 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.173429 Sep 18 06:28:13.799055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 245/499] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:13 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:14.279850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:14.281091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:14.288118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:14.288842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:14.289306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:14.290002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:14.290455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:14.290916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:14.488649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:14.492513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:14.502688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c2144dde-5461-42a3-b6b2-74ee4ed29f05 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.225689 Sep 18 06:28:14.503458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 246/500] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:14.608458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:14.609323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:14.620594 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:14.621157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:14.621690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:14.622809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:14.623511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:14.624130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:14.667407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bcd255a3-84e6-437f-8fa8-6a5ab994e181 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.065732 Sep 18 06:28:14.668628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 255/501] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:28:13 2022] POST /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 194 bytes in 1068 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:14.693678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:14.695513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:14.721955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:14.722478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:14.722948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:14.723829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:14.724087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:14.725630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:14.841838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:14.852814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-593232c2-d5dc-4a6a-ae21-d90e6c2d6cc0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1319 microversion: 2.1 time: 0.247398 Sep 18 06:28:14.867332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 247/502] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:14 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1319 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:14.886982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:14.887674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:14.901404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:14.902990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:14.903432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:14.904456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:14.926654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:14.927282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.012644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:15.029566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:15.031554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:15.033191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e51bd8d1-f7ea-4ed0-ad79-07f520ff002d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.363213 Sep 18 06:28:15.050806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 256/503] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:14 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:15.136848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:15.151856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c9517905-e2d5-448c-b6a9-cca2738c2e8f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.282991 Sep 18 06:28:15.152441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 248/504] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:14 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:15.527995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:15.528800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:15.529604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:15.530308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:15.537203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.537702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.538146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.538439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.538944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.539364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.539496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.539919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.540549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.541332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.541986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.542631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.574964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-de47946a-b39f-47ed-a3aa-9450c98b4ab5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd/ips" status: 200 len: 149 microversion: 2.1 time: 0.050605 Sep 18 06:28:15.582877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 249/505] 10.222.0.20 () {58 vars in 1248 bytes} [Sun Sep 18 06:28:15 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd/ips => generated 149 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:15.588922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2e3b9bc4-9d2a-401b-a62a-91136b41b8f4"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:15.591548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:15.598792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.599335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.599926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.600723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.601276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.601798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.718460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Invalid volume: volume 2e3b9bc4-9d2a-401b-a62a-91136b41b8f4 is already attached to instances: cb5ac07d-bc0f-455d-a133-6abb2e30f695 Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: POST /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd/os-volume_attachments HTTP/1.1 Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 74 Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:28:15.720534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {"volumeAttachment": {"volumeId": "2e3b9bc4-9d2a-401b-a62a-91136b41b8f4"}} Sep 18 06:28:15.722172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: volume 2e3b9bc4-9d2a-401b-a62a-91136b41b8f4 is already attached to instances: cb5ac07d-bc0f-455d-a133-6abb2e30f695 Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': '2131eb93-ac09-4fc4-991e-3a0cc04b5bbd', 'body': {'volumeAttachment': {'volumeId': '2e3b9bc4-9d2a-401b-a62a-91136b41b8f4'}}} Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 2e3b9bc4-9d2a-401b-a62a-91136b41b8f4 is already attached to instances: cb5ac07d-bc0f-455d-a133-6abb2e30f695 Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:28:15.723701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 2e3b9bc4-9d2a-401b-a62a-91136b41b8f4 is already attached to instances: cb5ac07d-bc0f-455d-a133-6abb2e30f695 Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:28:15.725145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:28:15.727676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:28:15.727676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:28:15.727676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:28:15.727676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 2e3b9bc4-9d2a-401b-a62a-91136b41b8f4 is already attached to instances: cb5ac07d-bc0f-455d-a133-6abb2e30f695 Sep 18 06:28:15.727676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:28:15.727676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 400 to user: Invalid volume: volume 2e3b9bc4-9d2a-401b-a62a-91136b41b8f4 is already attached to instances: cb5ac07d-bc0f-455d-a133-6abb2e30f695 {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:28:15.727676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a6bd5d7f-8173-4416-9a50-60e1e40dd52b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.137000 Sep 18 06:28:15.727676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 250/506] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:28:15 2022] POST /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd/os-volume_attachments => generated 174 bytes in 139 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:28:15.738351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:15.738905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:15.741308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:15.745076 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.745587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.746398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.747884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.748462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.748880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.751076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cb757e72-cac8-4983-8c59-38e243506d28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.226201 Sep 18 06:28:15.763119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 257/507] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:15 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:15.777608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Going to try to terminate instance {{(pid=120380) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:28:15.879674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b6eba550-7cf5-4b0d-9c21-80bf4583ea07 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 204 len: 0 microversion: 2.1 time: 0.144306 Sep 18 06:28:15.880874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 251/508] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:28:15 2022] DELETE /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:28:15.890261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:15.891297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:15.895335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:15.896294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:15.900633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.901153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.901590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.902307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.902782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.904732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.922293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.923462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.924028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:15.924873 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:15.925960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:15.926483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:16.117527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:16.127232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-add68703-46de-4a4d-913c-d50f8b147e34 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1681 microversion: 2.1 time: 0.241625 Sep 18 06:28:16.128647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 258/509] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:15 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1681 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:16.133982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:16.135212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:16.141691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:16.142436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:16.143238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:16.144188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:16.145572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:16.146204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:16.157055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:16.167178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d91ad1a5-9c02-4447-a8bf-64d3b410614c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1319 microversion: 2.1 time: 0.276435 Sep 18 06:28:16.178740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 252/510] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:15 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1319 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:16.178740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:16.178740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:16.187807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:16.188338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:16.189459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:16.190177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:16.190678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:16.192480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:16.382124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:16.393545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:16.395354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:16.397094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:16.398952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-22c4f5fc-7fd6-49d3-ba84-744dc30bf213 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.268530 Sep 18 06:28:16.399638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 259/511] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:16 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:16.425552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b3e20f55-4830-440a-b1b9-d63e2fa9ada7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.253464 Sep 18 06:28:16.426619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 253/512] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:16 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:16.779938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:16.781321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:16.788377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:16.789543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:16.790194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:16.791378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:16.791960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:16.793310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:16.984004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:17.006101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4537f13b-025f-405d-aad3-19ea5dab95d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.232633 Sep 18 06:28:17.006870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 260/513] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:17.158288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:17.158986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:17.161927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:17.169192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:17.169694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:17.170130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:17.170873 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:17.171339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:17.171761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:17.192572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:17.193386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:17.201110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:17.202024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:17.202776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:17.203785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:17.204629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:17.205288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:17.391068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:17.401686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ebc6c283-a4ed-4e27-b975-9465c98f40e5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1681 microversion: 2.1 time: 0.247670 Sep 18 06:28:17.418908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 254/514] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:17 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1681 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:17.431544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:17.432126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:17.438760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:17.439159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:17.439599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:17.440306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:17.440792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:17.441796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:17.456638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:17.483518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-50f80b5d-c970-46a4-b055-64a1dfdd1371 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1319 microversion: 2.1 time: 0.295933 Sep 18 06:28:17.501321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 261/515] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:17 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1319 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:17.519896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:17.523209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:17.531859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:17.532347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:17.532785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:17.533484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:17.533960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:17.534377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:17.659425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:17.668904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:17.672293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:17.674068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e44b09b6-cbd0-41a2-aa9b-5d43c9e629a5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.246404 Sep 18 06:28:17.682805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 255/516] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:17 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:17.733962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:17.743018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a18c0632-0b6f-40df-b64d-098b5e88200e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.240107 Sep 18 06:28:17.743851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 262/517] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:17 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:18.046938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:18.046938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:18.046938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.046938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.053342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.053342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.053342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.053342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.207476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:18.215809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6c361074-2d4c-47b9-89bf-fecac9c3dcb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1759 microversion: 2.1 time: 0.187764 Sep 18 06:28:18.216371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 256/518] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:18 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1759 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:18.435941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:18.436552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:18.444707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.445500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.445976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.446777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.447435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.447896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.517321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:18.518332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:18.520932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:18.521717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:18.529888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.530346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.531002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.531718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.532173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.532599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.582190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:18.621985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:18.631652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-022dfcca-9731-4d94-8f49-e45cbfe5cfbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1681 microversion: 2.1 time: 0.199711 Sep 18 06:28:18.632232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 263/519] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:18 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1681 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:18.693380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:18.702829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:18.703601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:18.710838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=91,nova_api:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:18.720697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-aee87911-2eb7-4414-b828-c23ccd71786c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1319 microversion: 2.1 time: 0.207130 Sep 18 06:28:18.725706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.726248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 257/520] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:18 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1319 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:18.726838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.727606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.728548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.729228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.729925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.767702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:18.768409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:18.777332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.781513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.782289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.783257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:18.784539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:18.784539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:18.973495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:18.986295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:18.990888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:18.992314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f8777f3f-3e4f-4005-b711-e282281ec8c7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1739 microversion: 2.20 time: 0.294925 Sep 18 06:28:18.994211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 264/521] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:18 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1739 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:19.009577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-5dc039aa-13fa-44c8-9acd-f4d5ce5eb3d1 req-60d92cd1-4f3a-49dc-96af-16447969e860 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7", "name": "network-vif-plugged", "status": "completed", "tag": "3b32ba19-f05b-4744-adab-ddcb74d679c1"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:19.025919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5dc039aa-13fa-44c8-9acd-f4d5ce5eb3d1 req-60d92cd1-4f3a-49dc-96af-16447969e860 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.026403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5dc039aa-13fa-44c8-9acd-f4d5ce5eb3d1 req-60d92cd1-4f3a-49dc-96af-16447969e860 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.026864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5dc039aa-13fa-44c8-9acd-f4d5ce5eb3d1 req-60d92cd1-4f3a-49dc-96af-16447969e860 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.028699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:19.039626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-5dc039aa-13fa-44c8-9acd-f4d5ce5eb3d1 req-60d92cd1-4f3a-49dc-96af-16447969e860 service nova] Creating event network-vif-plugged:3b32ba19-f05b-4744-adab-ddcb74d679c1 for instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 on hv-cinder-85535 Sep 18 06:28:19.043352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ffe982dc-45a5-463d-b27e-a2b0a125f9d0 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.280358 Sep 18 06:28:19.044721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-5dc039aa-13fa-44c8-9acd-f4d5ce5eb3d1 req-60d92cd1-4f3a-49dc-96af-16447969e860 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050332 Sep 18 06:28:19.045302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 258/522] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:18 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:19.057413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 265/523] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:28:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:19.228868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-d878288c-fbaf-4ce9-8041-385b2759afc6 req-deafdd8e-b6f0-4443-a2e4-8267306b31cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2131eb93-ac09-4fc4-991e-3a0cc04b5bbd", "name": "network-vif-deleted", "tag": "96889ca2-75ee-41de-b4a7-a0b675cf59ff"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:19.241846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:19.242745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:19.244946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d878288c-fbaf-4ce9-8041-385b2759afc6 req-deafdd8e-b6f0-4443-a2e4-8267306b31cb service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.246511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d878288c-fbaf-4ce9-8041-385b2759afc6 req-deafdd8e-b6f0-4443-a2e4-8267306b31cb service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.246992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d878288c-fbaf-4ce9-8041-385b2759afc6 req-deafdd8e-b6f0-4443-a2e4-8267306b31cb service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.250952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.251747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.252444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.253902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.254843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.255497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.278651 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-d878288c-fbaf-4ce9-8041-385b2759afc6 req-deafdd8e-b6f0-4443-a2e4-8267306b31cb service nova] Creating event network-vif-deleted:96889ca2-75ee-41de-b4a7-a0b675cf59ff for instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd on hv-cinder-85535 Sep 18 06:28:19.301673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-d878288c-fbaf-4ce9-8041-385b2759afc6 req-deafdd8e-b6f0-4443-a2e4-8267306b31cb service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073252 Sep 18 06:28:19.302936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 259/524] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:28:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:19.518652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:19.528410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0be1710e-1843-4f15-a993-f457fa8f9eab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1762 microversion: 2.1 time: 0.291160 Sep 18 06:28:19.529076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 266/525] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:19 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1762 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:19.657913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:19.663122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:19.671140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.671834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.672491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.673290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.673809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.674323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.729927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.730454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.731004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.731494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lazy-loading 'fault' on Instance uuid 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:28:19.757999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:19.767519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:19.777550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.778046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.778710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.784967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:19.787574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:19.788227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:19.815277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:19.832207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6862e26a-ecba-490b-a50c-c66d00bc9188 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1331 microversion: 2.1 time: 0.176880 Sep 18 06:28:19.834829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 260/526] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:19 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1331 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:19.959487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:19.968164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-83419e41-b071-4b64-82d2-18845237ebc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1319 microversion: 2.1 time: 0.219934 Sep 18 06:28:19.969078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 267/527] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:19 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1319 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:20.021660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:20.022462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:20.029021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:20.029585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:20.030086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:20.031579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:20.031891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:20.032387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:20.067381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:20.068477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:20.074950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:20.075780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:20.076977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:20.077971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:20.078662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:20.079312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:20.357609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:20.390320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:20.392276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:20.395661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d4c1ae42-d8ec-4168-8aab-21582b942db4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1721 microversion: 2.20 time: 0.377125 Sep 18 06:28:20.406923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:20.418190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 261/528] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:20 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1721 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:20.418888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2eeec437-dfea-4bc4-b3f1-5e62f0947a09 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.355760 Sep 18 06:28:20.419947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 268/529] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:20 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:20.861456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:20.861977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:20.870641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:20.871126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:20.871558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:20.872245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:20.872693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:20.873110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:20.891421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:20.907736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:20.908224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:20.908655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:20.909108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lazy-loading 'fault' on Instance uuid 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:28:20.952621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:20.960993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bc9e3d72-e2fe-40a0-ad4b-e30ebae98a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1331 microversion: 2.1 time: 0.103510 Sep 18 06:28:20.961574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 262/530] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:20 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1331 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:20.990913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:20.991575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:20.998207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:20.998724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:20.999157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:20.999857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.000308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.000727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:21.139291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:21.149441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0ca659c7-8ce5-436e-bcf1-26545dbcf883 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1509 microversion: 2.1 time: 0.162295 Sep 18 06:28:21.150481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 269/531] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:20 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1509 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:21.176135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0dd5997f-7321-4086-9598-6cca845c9907", "device": "/dev/sdb"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:21.178726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:21.184939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.185497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.186009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:21.186815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.187363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.187857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:21.424714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:21.425323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:21.431299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.431780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.432218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:21.432904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.433356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.433785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:21.563229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:21.571906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:21.573370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:21.574593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-463633bf-bacf-40b1-ad09-93e7b4a6627b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1721 microversion: 2.20 time: 0.153748 Sep 18 06:28:21.575150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 270/532] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:21 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:21.579478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:21.580087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:21.586512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.587051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.587493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:21.588185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.588640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.589119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:21.593081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:21.723089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:21.731255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b13dea31-8802-4787-bbe8-46c7cfd24bf1 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.155079 Sep 18 06:28:21.731847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 271/533] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:21 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:21.984924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:21.985427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:21.991410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.992000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.992315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:21.993002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:21.993453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:21.993878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:22.024427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:22.024904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:22.025335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:22.025829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lazy-loading 'fault' on Instance uuid 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:28:22.073572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:22.081828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-87da6006-1923-41c1-a6a0-f60d16553e21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 200 len: 1331 microversion: 2.1 time: 0.100405 Sep 18 06:28:22.082397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 272/534] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:21 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 1331 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:22.601389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:22.601917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:22.607845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:22.608328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:22.608794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:22.609480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:22.609929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:22.610403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:22.738393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:22.746507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:22.747974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:22.749145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c91d5ad5-8d47-45f3-a32d-db41d6e7639d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1721 microversion: 2.20 time: 0.152445 Sep 18 06:28:22.749705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 273/535] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:22 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:22.754313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:22.754958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:22.760906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:22.761455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:22.761953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:22.762742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:22.763294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:22.763773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:22.896769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:22.904670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-08866cb7-c593-4ab7-ac99-19bdbf2aa13a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.154014 Sep 18 06:28:22.905327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 274/536] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:22 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:22.910317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-1f206742-b99a-4b13-aa93-8215edb86dd1 req-768faf19-9661-46fe-811d-5063bcd3458d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ed3cfa3-6093-4c26-9877-01470acd2e16", "name": "network-vif-unplugged", "status": "completed", "tag": "84893f57-6d56-4075-9e20-5bd22c657e44"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:22.919962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1f206742-b99a-4b13-aa93-8215edb86dd1 req-768faf19-9661-46fe-811d-5063bcd3458d service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:22.920449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1f206742-b99a-4b13-aa93-8215edb86dd1 req-768faf19-9661-46fe-811d-5063bcd3458d service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:22.920879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1f206742-b99a-4b13-aa93-8215edb86dd1 req-768faf19-9661-46fe-811d-5063bcd3458d service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:22.933560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-1f206742-b99a-4b13-aa93-8215edb86dd1 req-768faf19-9661-46fe-811d-5063bcd3458d service nova] Creating event network-vif-unplugged:84893f57-6d56-4075-9e20-5bd22c657e44 for instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 on hv-cinder-85535 Sep 18 06:28:22.939486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-1f206742-b99a-4b13-aa93-8215edb86dd1 req-768faf19-9661-46fe-811d-5063bcd3458d service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032904 Sep 18 06:28:22.940083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 275/537] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:28:22 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:23.106237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:23.106998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:23.113775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:23.114256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:23.114721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:23.115405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:23.115858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:23.116278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:23.129566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd HTTP/1.1 Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:28:23.131162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:28:23.132748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '2131eb93-ac09-4fc4-991e-3a0cc04b5bbd'} Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:28:23.133573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:28:23.134868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:28:23.136005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:28:23.136005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-48f0292d-f5bb-4f43-a6dc-d3aa3e939a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 404 len: 111 microversion: 2.1 time: 0.031049 Sep 18 06:28:23.136005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 276/538] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:23 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 06:28:23.140831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cf494f65-1b70-446e-9085-8dd265762f4e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '.version_select at 0x7fe670563ca0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:23.160331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cf494f65-1b70-446e-9085-8dd265762f4e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-143579324" status: 202 len: 0 microversion: 2.1 time: 0.022881 Sep 18 06:28:23.160896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 277/539] 10.222.0.20 () {58 vars in 1218 bytes} [Sun Sep 18 06:28:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-143579324 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 06:28:23.586643 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a83d0219-cf48-4a43-95ac-b8d9f6541aa3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "POST /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.414947 Sep 18 06:28:23.587404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 263/540] 10.222.0.20 () {60 vars in 1323 bytes} [Sun Sep 18 06:28:21 2022] POST /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments => generated 194 bytes in 2417 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:23.775586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:23.776205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:23.782935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:23.783428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:23.783871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:23.784568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:23.785026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:23.785456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:23.930350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:23.931109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:23.939949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:23.941012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:23.941457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:23.942372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:23.943114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:23.943701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:23.955794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:23.965771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:23.967452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:23.968716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f8271863-b33d-45f6-bff5-ea7107ee8643 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1724 microversion: 2.20 time: 0.197229 Sep 18 06:28:23.969799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 278/541] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:23 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1724 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:23.974672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:23.975356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:23.981344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:23.982039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:23.983401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:23.984470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:23.985080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:23.985818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.157258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lazy-loading 'services' on Instance uuid cb5ac07d-bc0f-455d-a133-6abb2e30f695 {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:28:24.162344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:24.172139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-50c34d2c-7859-41fc-a36a-2e5ea4ae6704 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.247742 Sep 18 06:28:24.182665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 264/542] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:23 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:24.182665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "15273f08-c5c3-4dc4-9f2f-b809495a2590"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:24.191513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:24.196570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.197132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.197589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.198462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.198774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.199206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.322232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c1c97d27-30e5-4b51-ae84-949b1cd98944 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments/2e3b9bc4-9d2a-401b-a62a-91136b41b8f4" status: 202 len: 0 microversion: 2.1 time: 0.351013 Sep 18 06:28:24.322851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 279/543] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:28:23 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments/2e3b9bc4-9d2a-401b-a62a-91136b41b8f4 => generated 0 bytes in 352 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:28:24.331847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:24.335651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:24.341753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.342232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.342692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.343383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.343833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.344254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.377223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-472750e7-5101-44f2-b73a-60f08f5b4d18 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049838 Sep 18 06:28:24.377790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 280/544] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:24 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:28:24.533205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5413eaf2-7dcb-49ee-852c-6eb560324173 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.356316 Sep 18 06:28:24.533987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 265/545] 10.222.0.20 () {62 vars in 1364 bytes} [Sun Sep 18 06:28:24 2022] POST /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 188 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:28:24.582196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:24.585550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:24.594813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.595466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.596088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.597660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.598320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.599171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.634797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lazy-loading 'flavor' on Instance uuid a8254cee-e629-4d82-988c-708bac734c1e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:28:24.688447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:24.689053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:24.695711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.696507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.697491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.698676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.699410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.700041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.750330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120380) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 18 06:28:24.808376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a767d429-2de6-499d-9a0f-92b92426ba9a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/action" status: 200 len: 32 microversion: 2.1 time: 0.231752 Sep 18 06:28:24.808990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 281/546] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:28:24 2022] POST /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/action => generated 32 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 06:28:24.819295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:24.819927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:24.825536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1dcc3753-9be2-4ce2-a65d-0b1d7e6edaf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/action" status: 202 len: 0 microversion: 2.20 time: 0.140992 Sep 18 06:28:24.826023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.826425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 266/547] 10.222.0.20 () {62 vars in 1319 bytes} [Sun Sep 18 06:28:24 2022] POST /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/action => generated 0 bytes in 142 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 18 06:28:24.829556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.830015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.830802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.831307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.831756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.834932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:24.835541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:24.841710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.842284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.842748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:24.843437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:24.843883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:24.844297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:25.069328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:25.082673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:25.092585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ecccd86d-6eed-4cfe-bc0e-9b1832f31762 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.269324 Sep 18 06:28:25.094142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:25.094465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 282/548] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:24 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:25.096194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:25.099546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9c7132a5-417e-4bce-90cb-af33b33e2958 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1750 microversion: 2.20 time: 0.268121 Sep 18 06:28:25.111490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 267/549] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:24 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1750 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:25.201642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:25.202854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:25.213627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:25.214352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:25.214842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:25.215563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:25.215994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:25.216463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:25.393130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:25.399341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:25.408101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:25.408597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:25.409075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:25.410064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:25.410523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:25.410969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:25.456799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-adb0473a-9ae3-4295-b9a8-7e44640cc0e3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068963 Sep 18 06:28:25.466905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 268/550] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:25 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:25.747134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:25.758120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-48be779f-1eae-4a55-ad05-4a2f3bae097c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.562797 Sep 18 06:28:25.758927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 283/551] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:25 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:26.117408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:26.118007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:26.124435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:26.124946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:26.125377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:26.125763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:26.126059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:26.126364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:26.126507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:26.126958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:26.133076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:26.133558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:26.133994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:26.134710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:26.135198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:26.135623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:26.318317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:26.322195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:26.327031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-246a8fd3-468d-4bc4-b4fd-22af72ff8b3b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.213346 Sep 18 06:28:26.327617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 269/552] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:26.330723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:26.332161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:26.333307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-72409d82-2bcb-428c-b813-58e57d1f3422 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.211363 Sep 18 06:28:26.333852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 284/553] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:26 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:26.478874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:26.481054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:26.487774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:26.488196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:26.488627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:26.489313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:26.489767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:26.490190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:26.519811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f0ab7ab4-29b1-4041-888f-266036f73a51 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044979 Sep 18 06:28:26.520353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 270/554] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:26 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:26.782473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:26.783116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:26.789503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:26.789979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:26.790410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:26.791119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:26.791570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:26.791993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:26.921418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:26.929554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-71ce35fb-f606-45f2-8e71-9f64cc54a804 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1675 microversion: 2.1 time: 0.150915 Sep 18 06:28:26.930123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 285/555] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:26 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1675 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:27.350302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:27.350962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:27.357020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:27.357458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:27.357623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:27.357942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:27.358388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:27.359117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:27.359575 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:27.360008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:27.363924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:27.364398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:27.364827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:27.365504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:27.365948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:27.366362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:27.554272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:27.558314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:27.562590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c12d77b6-4509-4f68-9677-257335d87f00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.216101 Sep 18 06:28:27.563159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 271/556] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:27.569247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:27.571389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:27.576334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:27.577249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:27.577898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:27.578478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:27.578696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:27.579673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:27.579811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-59c1e0b8-a7e6-4501-b43c-9c56fb945232 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.226526 Sep 18 06:28:27.580509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:27.580652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 286/557] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:27 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:27.582182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:27.613145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-aab5a0f4-b212-44e7-a497-ac2c347d4892 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048788 Sep 18 06:28:27.613639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 272/558] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:27 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:27.952426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:27.953035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:27.959598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:27.960086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:27.960532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:27.961222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:27.961676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:27.962102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:27.964766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:28.090232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:28.097920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-40da305b-bef5-4491-8702-1270e98a6f50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1652 microversion: 2.1 time: 0.149405 Sep 18 06:28:28.098481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 287/559] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:27 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1652 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:28.120054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e4e0f3fa-0842-44ab-85cc-c4a488e49181 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:28.125919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4e0f3fa-0842-44ab-85cc-c4a488e49181 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.126394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4e0f3fa-0842-44ab-85cc-c4a488e49181 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.126857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4e0f3fa-0842-44ab-85cc-c4a488e49181 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.149497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e4e0f3fa-0842-44ab-85cc-c4a488e49181 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Going to try to start instance {{(pid=120380) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 18 06:28:28.153903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=7,nova_cell1:INSERT=5 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:28.189900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e4e0f3fa-0842-44ab-85cc-c4a488e49181 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/action" status: 202 len: 0 microversion: 2.1 time: 0.073594 Sep 18 06:28:28.191034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 273/560] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:28:28 2022] POST /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/action => generated 0 bytes in 75 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:28:28.199060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:28.199809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:28.207328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.207812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.208248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.209014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.209488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.209910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.417627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:28.425807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a865a575-469d-41a0-8c4e-c4728bb75327 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1661 microversion: 2.1 time: 0.230462 Sep 18 06:28:28.426374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 288/561] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:28 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1661 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:28.587359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:28.588053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:28.594180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.594689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.595124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.595812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.596260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.596679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.605342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:28.606093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:28.612944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.613500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.613937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.614819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.615498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.616086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.896617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:28.905074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b6a923b7-9fca-4ef4-bdc0-0353a8e8f46f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.322584 Sep 18 06:28:28.905644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 274/562] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:28 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:28.911047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:28.913341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:28.923238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.923913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.926968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.928088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:28.928913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:28.929552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:28.930641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:28.939491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:28.940993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:28.942152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-022a45a6-08d9-45b2-a30a-b5ad1707c08d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.340829 Sep 18 06:28:28.965265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 289/563] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:28 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:28.970721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-38e98980-7fb5-4521-8ac7-0ff94bd5b7c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063786 Sep 18 06:28:28.972373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 275/564] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:28 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:29.449290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:29.449891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:29.456480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:29.456957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:29.457389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:29.458096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:29.458543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:29.458988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:29.585239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:29.592856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-862ff0fd-7d65-4343-83f5-b9f6b4460c0a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1661 microversion: 2.1 time: 0.147460 Sep 18 06:28:29.593418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 290/565] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:29 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1661 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:29.931051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:29.931684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:29.937981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:29.938459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:29.938923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:29.939601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:29.940051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:29.940469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:29.972293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:29.972903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:29.979743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:29.979939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:29.980428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:29.981150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:29.981714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:29.982228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:30.119711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:30.127943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1528f8f0-e534-47e0-b2c5-478664f743cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.200703 Sep 18 06:28:30.128703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 276/566] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:30.148038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:30.150170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:30.156096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:30.156578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:30.157011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:30.157698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:30.158147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:30.158568 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:30.158726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:30.168331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:30.169939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:30.171187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-45757767-6f33-4e76-83cb-70defc708e0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.203778 Sep 18 06:28:30.171751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 291/567] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:29 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:30.188307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5dd0bc89-3cc9-4f4c-8d9c-fe76d8b3b68c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043951 Sep 18 06:28:30.188780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 277/568] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:30 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:30.615696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:30.616301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:30.622709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:30.623186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:30.623615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:30.624290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:30.624736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:30.625151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:30.751211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:30.759676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1b394cc3-f1a0-4ad2-9d1f-63f1aafed8ff tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1665 microversion: 2.1 time: 0.148027 Sep 18 06:28:30.760235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 292/569] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:30 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1665 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:31.154127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:31.154953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:31.161158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:31.161644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:31.162079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:31.162794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:31.163247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:31.163667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:31.199682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:31.200504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:31.208138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:31.208942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:31.209084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:31.209833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:31.210289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:31.210856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:31.363097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:31.371499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-686ad8a8-ca0f-4f22-96c1-82a88966e3aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.221052 Sep 18 06:28:31.372688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 278/570] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:31.386881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:31.389737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:31.394090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:31.396341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:31.396821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:31.397283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:31.397971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:31.398417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:31.398860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:31.402845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:31.404293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:31.405456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f3bae0dd-844e-4c10-a38b-a412a8775fae tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.212167 Sep 18 06:28:31.406012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 293/571] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:31 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:31.428014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-139f836c-e41c-4fc1-a2fc-7c57420155eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053759 Sep 18 06:28:31.428565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 279/572] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:31 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:32.396608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:32.397214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:32.403639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:32.404141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:32.404573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:32.405283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:32.405759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:32.406206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:32.431331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:32.432039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:32.438460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:32.439051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:32.439550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:32.440925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:32.441441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:32.441865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:32.586706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:32.595179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8ad45204-2a6a-4299-af9b-a2caf87bbe13 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.202410 Sep 18 06:28:32.595787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 294/573] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:32 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:32.601102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:32.603376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:32.609893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:32.610378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:32.610865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:32.611561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:32.612605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:32.613029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:32.613633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:32.622468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:32.624008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:32.625206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-40f987b0-c8b3-4af6-9bbe-04a931294df5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.197732 Sep 18 06:28:32.625759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 280/574] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:32 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:32.643451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ea874990-3402-470b-995a-98d1ecf344d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046112 Sep 18 06:28:32.644003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 295/575] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:32 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:33.618419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:33.618988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:33.625368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:33.625845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:33.626277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:33.626992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:33.627950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:33.628372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:33.650077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:33.650800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:33.657162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:33.657645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:33.658111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:33.658896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:33.659351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:33.659771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:33.819399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:33.827762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ff5a577b-1deb-4889-a883-dae19a35da91 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.213139 Sep 18 06:28:33.828335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 281/576] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:33 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:33.830374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:33.833055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:33.835223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:33.839858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:33.841283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:33.841427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:33.841772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:33.842219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:33.842444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-109a5518-8c61-48f1-b720-61a23b76a1c0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.196203 Sep 18 06:28:33.842943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:33.843046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 296/577] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:33 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:33.843469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:33.843968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:33.873561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8d80a06c-2f58-4751-b42d-4cd84b7d3eaf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044159 Sep 18 06:28:33.874064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 282/578] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:33 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:34.850646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:34.851252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:34.857728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:34.858207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:34.858668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:34.859349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:34.859820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:34.860240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:34.866819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:34.867541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:34.873913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:34.874392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:34.874863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:34.875549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:34.875997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:34.876419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:35.044667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:35.053362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6b13ced3-9583-4046-a053-71e1de569b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.206509 Sep 18 06:28:35.053914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 297/579] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:35.058430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:35.060594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:35.066297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:35.066809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:35.067241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:35.067352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:35.067918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:35.068370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:35.068787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:35.076054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:35.077531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:35.078726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a45c01c6-688f-4de0-9cb1-e1a2319a82ef tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215981 Sep 18 06:28:35.079282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 283/580] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:34 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:35.098495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-51f99263-2aca-473a-82e3-8b8c5546aa12 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043552 Sep 18 06:28:35.099064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 298/581] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:35 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:36.076892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:36.077504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:36.083916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:36.084396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:36.084832 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:36.085520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:36.086018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:36.086431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:36.104297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:36.104906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:36.111381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:36.112355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:36.112788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:36.113477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:36.113930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:36.114355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:36.285547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:36.288622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:36.294309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:36.295793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:36.297013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ee3ad0b5-c1eb-42cf-b068-531ec0654d3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.224051 Sep 18 06:28:36.297150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-325bbdfd-bc60-42ac-956b-adac8b995434 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.196640 Sep 18 06:28:36.297936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 284/583] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:36.298062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 299/583] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:36 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:36.303064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:36.305156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:36.310992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:36.311473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:36.311909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:36.312590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:36.313043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:36.313462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:36.341779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-53e7468e-d740-4112-8150-f6f319d05dbe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043050 Sep 18 06:28:36.342328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 300/584] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:36 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:37.321999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:37.322541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:37.322948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:37.323034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:37.329047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:37.329179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:37.329535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:37.329690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:37.329969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:37.330078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:37.331328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:37.331462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:37.331783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:37.331888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:37.332209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:37.332312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:37.532644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:37.536071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:37.541346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d1caf8b5-c07d-46fb-8a1e-3fc8a1de5bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.223182 Sep 18 06:28:37.541992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 285/585] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:37 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:37.545226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:37.546852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:37.546972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:37.548110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fb6941a6-3edf-40bd-b1d4-1ab9abe25755 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.229649 Sep 18 06:28:37.549448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 301/586] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:37 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:37.549633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:37.555824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:37.556307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:37.556884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:37.557734 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:37.558295 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:37.558770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:37.587728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e1b26569-a088-4d96-ad0e-65d995f06678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044646 Sep 18 06:28:37.588204 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 286/587] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:37 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:38.565094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:38.565705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:38.572175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:38.572689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:38.573109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:38.573498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:38.573719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:38.573821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:38.574246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:38.574701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:38.580140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:38.580616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:38.581056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:38.581740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:38.582189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:38.582634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:38.768260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:38.772180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:38.776570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-30642bfd-7318-4462-917f-14356da32030 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.215245 Sep 18 06:28:38.777137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 302/588] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:38 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:38.780801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:38.781713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:38.782248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:38.783935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-15276348-f9a9-49ce-af27-e761f14831fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214220 Sep 18 06:28:38.784646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:38.784770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 287/589] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:38 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:38.790654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:38.791210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:38.791647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:38.792331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:38.792776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:38.793198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:38.818298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:38.823421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-07bcba95-7d81-41d4-a5f3-edcdea393147 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045341 Sep 18 06:28:38.823971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 303/590] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:38 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:39.802352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:39.803010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:39.809066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:39.809496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:39.809706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:39.809890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:39.810321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:39.811031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:39.811486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:39.811935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:39.816219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:39.816702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:39.817145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:39.817870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:39.818327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:39.818780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:40.005334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:40.008750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:40.013656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d60a5363-3862-47d7-baa7-c76174cc3878 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.215098 Sep 18 06:28:40.014227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 288/591] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:39 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:40.018365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:40.019429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:40.019830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:40.020989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-990e01f3-bd13-427b-b2e9-3e24c2879685 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215861 Sep 18 06:28:40.021603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:40.021916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 304/592] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:39 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:40.027627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:40.028103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:40.028532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:40.029209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:40.029663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:40.030081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:40.048266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:40.062931 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f4834e57-f8fc-40d0-b576-961c9969dcbb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047559 Sep 18 06:28:40.063414 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 289/593] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:40 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:41.039559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:41.040166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:41.046267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:41.046736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:41.046897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:41.047010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:41.047439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:41.048128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:41.048647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:41.049659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:41.054512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:41.055015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:41.055562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:41.056272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:41.056749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:41.057208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:41.240323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:41.244257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:41.248721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4eef03da-184b-4905-9fd8-aa57f7da1c8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.212961 Sep 18 06:28:41.249303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 305/594] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:41.253237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:41.254359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:41.255097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:41.256410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8017b2cd-6e23-49fe-a510-31fef43d74d9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.213959 Sep 18 06:28:41.256778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:41.257524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 290/595] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:41 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:41.262900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:41.263387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:41.263825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:41.264511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:41.264965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:41.265391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:41.295002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f652d977-b6d2-4976-80c6-1881e6aee2d5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044693 Sep 18 06:28:41.295556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 306/596] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:41 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:42.273617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:42.274303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:42.281034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:42.281523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:42.281959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:42.282151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:42.282675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:42.282782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:42.283629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:42.284055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:42.291183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:42.291671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:42.292109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:42.292797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:42.293251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:42.293722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:42.467331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:42.476981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:42.478426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:42.479644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-803310e2-4005-46b4-b3a1-4d8495d32c9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.201334 Sep 18 06:28:42.480333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 307/597] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:42 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:42.485095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:42.487871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:42.494188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:42.494695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:42.495138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:42.495823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:42.496274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:42.496696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:42.501327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:42.511986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-20ba6bf1-e45c-4fe8-ae42-6867d2a9d7e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.241827 Sep 18 06:28:42.522269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 291/598] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:42.528569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d464062c-38d9-432c-9441-b10b0a7cafe4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047082 Sep 18 06:28:42.529072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 308/599] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:42 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:43.506064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:43.506696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:43.513035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:43.513512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:43.513943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:43.514671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:43.515102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:43.515521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:43.535356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:43.536007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:43.542279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:43.542800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:43.543234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:43.543909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:43.544356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:43.544783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:43.704368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:43.713347 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:43.714839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:43.716071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e052a59f-8e54-4634-9c0b-8ea8453dfe12 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214767 Sep 18 06:28:43.716572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 292/600] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:43 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:43.721230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:43.723369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:43.726796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:43.729213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:43.729704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:43.730213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:43.730944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:43.731400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:43.731819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:43.735230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-beddfef8-d7ac-44ba-9639-8636a1deff63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.203634 Sep 18 06:28:43.735795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 309/601] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:43.763520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-93fabaed-791f-41a4-838d-893a79b634cd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045789 Sep 18 06:28:43.764067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 293/602] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:43 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:44.742117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:44.742764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:44.749260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:44.749747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:44.750187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:44.750920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:44.751380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:44.751806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:44.760218 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:44.760834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:44.767262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:44.767778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:44.768212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:44.768932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:44.769409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:44.769837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:44.950074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:44.953928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:44.959193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:44.960636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:44.961788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-15c3ff46-0bf0-44fc-b2a0-5c65e6df012a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.223587 Sep 18 06:28:44.962334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 310/603] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:44 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:44.962619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-10acc49b-91ad-4bd4-b94f-79d32ba2b52b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.206196 Sep 18 06:28:44.963518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 294/604] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:44.967530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:44.969663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:44.976103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:44.976583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:44.977019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:44.977708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:44.978158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:44.978604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:45.008342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-786b99f2-5ac1-4922-9e6c-5dbbf15e12cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044989 Sep 18 06:28:45.008897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 311/605] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:44 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:45.986415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:45.986993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:45.987373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:45.987485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:45.989602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:45.994078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:45.994571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:45.994892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:45.995035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:45.995370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:45.995739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:45.995887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:45.996198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:45.996530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:45.996646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:45.996974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:45.997416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:46.190538 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:46.193729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:46.199091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8f5bdf52-7e3f-4122-b0d1-f619a2aed503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.216468 Sep 18 06:28:46.199745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 295/606] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:45 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:28:46.202535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:46.204085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:46.205260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:46.205880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e378c3d0-2323-4834-9e67-06b0e928f207 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.223050 Sep 18 06:28:46.206637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 312/607] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:45 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:46.207470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:46.213210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:46.213690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:46.214121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:46.214824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:46.215277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:46.215695 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:46.247087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-327377f1-a7d5-4865-9554-01eed037da01 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046148 Sep 18 06:28:46.247566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 296/608] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:46 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:47.223423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:47.224027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:47.230649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:47.231100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:47.231297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:47.231622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:47.232154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:47.232962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:47.233479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:47.233854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:47.233967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:47.239230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:47.239713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:47.240144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:47.240828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:47.241277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:47.241697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:47.422263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:47.429103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:47.430461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-14b87fd7-aed3-4de6-bd12-0caf5f16f873 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.211511 Sep 18 06:28:47.431061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 313/609] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:47.435636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:47.437750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:47.438144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:47.439619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:47.440788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f9acb620-9d43-4653-bb6f-e5e5a5a3dece tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214180 Sep 18 06:28:47.445495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 297/610] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:47 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:47.445628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:47.445628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:47.445628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:47.445628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:47.445946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:47.446224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:47.476498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f1bab7a9-756e-46da-8080-1ebacf9bcf7a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044343 Sep 18 06:28:47.477002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 314/611] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:47 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:48.453088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:48.453703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:48.460528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:48.460749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:48.461167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:48.461861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:48.462308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:48.462768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:48.466260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:48.466920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:48.473413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:48.473896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:48.474331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:48.475056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:48.475505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:48.475928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:48.657242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:48.661816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:48.666120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4a526cd1-5e45-4822-8b5f-e0550df3189c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.216799 Sep 18 06:28:48.666713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 298/612] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:48.670708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:48.671251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:48.672156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:48.673523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f2a1328c-d136-4cda-a0dd-d6ed1ab7854e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.211788 Sep 18 06:28:48.673642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:48.674071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 315/613] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:48 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:48.679491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:48.679997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:48.680629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:48.681318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:48.681914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:48.682338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:48.711198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6a0df3ce-2dec-415c-92af-de0995f84ba7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043504 Sep 18 06:28:48.711748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 299/614] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:48 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:49.689363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:49.690851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:49.696909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:49.697388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:49.697578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:49.697984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:49.698194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:49.698493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:49.698967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:49.699386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:49.704518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:49.704997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:49.705493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:49.706172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:49.706651 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:49.707079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:49.888907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:49.892969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:49.897183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0ce8d50c-77a8-4e87-9607-e0a86dfddc6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.211800 Sep 18 06:28:49.897747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 316/615] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:49.901493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:49.902948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:49.903257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:49.904096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8f451acb-b014-4475-96f6-ccf8b1b0dae7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.210519 Sep 18 06:28:49.904797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 300/616] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:49 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:49.905505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:49.911933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:49.912394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:49.912825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:49.913503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:49.913947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:49.914364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:49.943457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f8df6007-be62-4079-bbd2-c76d50d26b25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044675 Sep 18 06:28:49.943933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 317/617] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:49 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:50.920932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:50.921523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:50.928013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:50.928494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:50.928929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:50.929320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:50.929736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:50.929927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:50.930042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:50.930465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:50.936185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:50.936665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:50.937096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:50.937782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:50.938232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:50.938867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:51.121876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:51.124764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:51.130890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ba1de3f7-a564-45dd-8a2a-b0849b9e91c9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.213779 Sep 18 06:28:51.131461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 301/618] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:51.133892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:51.135363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:51.136020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:51.136502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34fb107b-e964-490d-911a-3f0d2d792e17 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.211109 Sep 18 06:28:51.137044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 318/619] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:50 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:51.138140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:51.144745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:51.145227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:51.145659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:51.146349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:51.146829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:51.147252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:51.176138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9868e51c-e288-41e9-8c05-12842ca6d222 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043625 Sep 18 06:28:51.176679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 302/620] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:51 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:52.155200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:52.155806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:52.160631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:52.161237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:52.162259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:52.162778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:52.163208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:52.163886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:52.164335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:52.164752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:52.167238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:52.167716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:52.168148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:52.168829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:52.169274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:52.169689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:52.363083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:52.366925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:52.371535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34f1bd87-0f2c-42b1-9773-9fb4400d0442 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.220175 Sep 18 06:28:52.372109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 319/621] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:52 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:52.375848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:52.376705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:52.377759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:52.379201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d1eac0d4-ae34-4292-9c6f-6cbaa76a3920 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.222458 Sep 18 06:28:52.379613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:52.385352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:52.385835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:52.386273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:52.386983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:52.387439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:52.387864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:52.390817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 303/622] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:52 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:52.402158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:52.418246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b1424721-0daa-4450-b99f-045d5a893020 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045108 Sep 18 06:28:52.418769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 320/623] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:52 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:53.396244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:53.396859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:53.403344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:53.403827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:53.404259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:53.404895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:53.405288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:53.405382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:53.405487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:53.405781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:53.412395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:53.412870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:53.413302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:53.413989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:53.414433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:53.414880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:53.596340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:53.599623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:53.604681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-529b99ac-aff5-40ec-851a-06d922d3a19c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.212707 Sep 18 06:28:53.605332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 304/624] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:53.608150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:53.609587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:53.610122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:53.610773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d6029467-905f-4456-a4e3-b9d66d3349ea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.209636 Sep 18 06:28:53.611486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 321/625] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:53 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:53.612604 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:53.618421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:53.619001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:53.619496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:53.620252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:53.620818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:53.621320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:53.651846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-06a80d4b-a0bc-4d82-a979-a0a75f80be42 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045393 Sep 18 06:28:53.652319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 305/626] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:53 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:54.627918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:54.628785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:54.635238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:54.635402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:54.635785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:54.636007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:54.636126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:54.636801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:54.637254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:54.637675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:54.642233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:54.643185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:54.643619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:54.644330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:54.644785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:54.645206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:54.649123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:54.829140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:54.832793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:54.837860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c3eb751e-3d5d-406e-b880-1790214be7ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.213781 Sep 18 06:28:54.838650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 322/627] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:54.841504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:54.843002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:54.843213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:54.844192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fcc47c43-fd56-4062-960b-b64817cfe424 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.212593 Sep 18 06:28:54.844863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 306/628] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:54 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:54.845373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:54.851636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:54.852110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:54.852541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:54.853219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:54.853663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:54.854080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:54.883202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b6f6a45a-9907-444e-9527-a67ec1a7dd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043481 Sep 18 06:28:54.883752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 323/629] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:54 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:55.861849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:55.862517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:55.869261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:55.869487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:55.869895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:55.870096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:55.870197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:55.870857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:55.871309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:55.871748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:55.876426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:55.876907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:55.877343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:55.878027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:55.878763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:55.879183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:56.094119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:56.098558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:56.103309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-34f32d8a-2f4d-4b6f-8571-0b1f65dd8ea3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.245963 Sep 18 06:28:56.103877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 307/630] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:56.107249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:56.108522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:56.108714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:56.109833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-68311ad1-bd7d-4859-805e-06fc1c8cfbbd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.245142 Sep 18 06:28:56.110518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 324/631] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:55 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:28:56.110839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:56.117212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:56.117640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:56.118074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:56.118791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:56.119248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:56.119669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:56.148959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2c7dd27d-4941-4268-b6e7-ea3845b16f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044070 Sep 18 06:28:56.149508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 308/632] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:56 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:57.127076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:57.127683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:57.134355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:57.134867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:57.135185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:57.135585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:57.135798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:57.135973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:57.136418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:57.136842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:57.143081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:57.143592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:57.144024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:57.144707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:57.145155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:57.145570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:57.330979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:57.334446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:57.339227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7baeda57-8690-4424-9bed-b11ebd5e8583 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.215977 Sep 18 06:28:57.339795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 325/633] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:57.354729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:57.354875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:57.354993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:57.354993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-aef74747-280e-48d4-9ae8-fe4ed1cccdab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214722 Sep 18 06:28:57.355173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:57.355263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 309/634] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:57 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:57.355353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:57.355353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:57.355353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:57.355353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:57.355706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:57.355706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:57.383025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e1133b1c-9393-41e3-946a-da944964fd6e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042217 Sep 18 06:28:57.383570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 326/635] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:57 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:58.362489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:58.363123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:58.369876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:58.370360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:58.370682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:58.371095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:58.371280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:58.371500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:58.371958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:58.372385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:58.378461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:58.378974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:58.379405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:58.380084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:58.380530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:58.380946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:58.805116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:58.808113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:58.813588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-627ce32f-5a01-45fe-a60e-9c7dcd7e6bd4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.454956 Sep 18 06:28:58.814202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 310/636] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:58 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:28:58.818791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:58.820891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:58.826095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:58.827567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:28:58.828734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9a46cfc5-d8c8-4fef-bb29-185c5c16978a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.462125 Sep 18 06:28:58.829072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:58.829533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 327/637] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:58 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:28:58.829670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:58.830107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:58.830854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:58.831307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:58.831726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:58.863535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4f298fda-93ea-4062-9972-2b278ccd8e60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048293 Sep 18 06:28:58.864075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 311/638] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:28:58 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:58.992234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1324081990"}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:58.992708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:58.998587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:58.999083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:58.999516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:59.000189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:59.000634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:59.001047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:59.127179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.network.security_group_api [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Removing security group ec5d246c-cc78-42f2-9d89-dd53ee15f560 from port 1956a12e-1f36-454a-baa5-f949db978b94 Sep 18 06:28:59.436481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f8aa5a79-229c-43b2-af35-883c2193a2f1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/action" status: 202 len: 0 microversion: 2.1 time: 0.448126 Sep 18 06:28:59.437657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 328/639] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:28:58 2022] POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/action => generated 0 bytes in 450 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:28:59.447678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-63375b99-4abf-46db-ae21-adbfcb36ab55 req-d1f4f406-bf3f-4c24-80f8-250bd4e11faf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b4bf8fc2-b57b-4198-939c-bd88739c785e", "tag": "1956a12e-1f36-454a-baa5-f949db978b94"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:28:59.460383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-63375b99-4abf-46db-ae21-adbfcb36ab55 req-d1f4f406-bf3f-4c24-80f8-250bd4e11faf service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:59.461382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-63375b99-4abf-46db-ae21-adbfcb36ab55 req-d1f4f406-bf3f-4c24-80f8-250bd4e11faf service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:59.461991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-63375b99-4abf-46db-ae21-adbfcb36ab55 req-d1f4f406-bf3f-4c24-80f8-250bd4e11faf service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:59.476091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-63375b99-4abf-46db-ae21-adbfcb36ab55 req-d1f4f406-bf3f-4c24-80f8-250bd4e11faf service nova] Creating event network-changed:1956a12e-1f36-454a-baa5-f949db978b94 for instance b4bf8fc2-b57b-4198-939c-bd88739c785e on hv-cinder-85535 Sep 18 06:28:59.482444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-63375b99-4abf-46db-ae21-adbfcb36ab55 req-d1f4f406-bf3f-4c24-80f8-250bd4e11faf service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038296 Sep 18 06:28:59.483818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 312/640] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:28:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:28:59.836637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:59.837299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:59.844553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:59.845717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:59.846191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:59.847165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:59.847901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:59.848523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:59.859521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:28:59.860344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:28:59.868759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:59.869356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:59.869895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:28:59.870955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:28:59.871482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:28:59.872134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:00.039201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:00.048933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7af17f38-a11d-4ce2-a7ed-d65332ba05bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.216115 Sep 18 06:29:00.049623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 329/641] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:28:59 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:00.054920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:00.057254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:00.064094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:00.064666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:00.065174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:00.066403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:00.066527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:00.067035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:00.076500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:00.087828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:00.089895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:00.091381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-01c2d946-fddf-49d6-8a75-b7bb6dd895b1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.240142 Sep 18 06:29:00.101774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dfa2c511-0773-4aff-8f72-efb3631c67fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050547 Sep 18 06:29:00.102621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 330/642] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:00 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:00.106852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 313/643] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:28:59 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:00.762868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:00.763586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:00.765760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-416dcc74-1c63-4f77-a45b-230c002239a6 req-c7d52960-20a8-4388-8d98-2de34d68e1ce service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b4bf8fc2-b57b-4198-939c-bd88739c785e", "tag": "1956a12e-1f36-454a-baa5-f949db978b94"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:00.770331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:00.771548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:00.772060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:00.772828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:00.773409 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:00.773890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:00.775817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-416dcc74-1c63-4f77-a45b-230c002239a6 req-c7d52960-20a8-4388-8d98-2de34d68e1ce service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:00.776459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-416dcc74-1c63-4f77-a45b-230c002239a6 req-c7d52960-20a8-4388-8d98-2de34d68e1ce service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:00.777072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-416dcc74-1c63-4f77-a45b-230c002239a6 req-c7d52960-20a8-4388-8d98-2de34d68e1ce service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:00.788008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-416dcc74-1c63-4f77-a45b-230c002239a6 req-c7d52960-20a8-4388-8d98-2de34d68e1ce service nova] Creating event network-changed:1956a12e-1f36-454a-baa5-f949db978b94 for instance b4bf8fc2-b57b-4198-939c-bd88739c785e on hv-cinder-85535 Sep 18 06:29:00.793164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-416dcc74-1c63-4f77-a45b-230c002239a6 req-c7d52960-20a8-4388-8d98-2de34d68e1ce service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031011 Sep 18 06:29:00.793910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 314/644] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:29:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:00.857022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:29:00.930802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-952e2170-c8fa-48a9-a9b6-95e741f3e506 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 202 len: 0 microversion: 2.1 time: 0.171733 Sep 18 06:29:00.931635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 331/645] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:29:00 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 0 bytes in 173 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:29:00.940998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:00.943355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:00.949529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:00.950098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:00.950633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:00.951331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:00.951789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:00.952224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:00.980856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d2138bf5-dbb7-4ce1-94e2-4213336d4ab3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044632 Sep 18 06:29:00.981517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 315/646] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:00 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:01.072850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:01.073585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:01.079968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:01.080455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:01.080895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:01.081587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:01.082105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:01.082533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:01.132727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:01.135596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:01.142241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:01.143304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:01.143304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:01.144362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:01.145764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:01.146360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:01.180575 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8187aba7-6bd8-4f98-9dc1-03366b8f074f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066340 Sep 18 06:29:01.181284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 316/647] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:01 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:01.190185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:01.190764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:01.197014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:01.197576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:01.198084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:01.198890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:01.199436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:01.199937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:01.224482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:01.234303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c1913f1f-da14-4ee7-bb22-0c1716932f3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.165226 Sep 18 06:29:01.238426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 332/648] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:01.332108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:01.340871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:01.342329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:01.343514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0b3d5ca4-4543-4222-9178-da39bc38b007 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157281 Sep 18 06:29:01.344066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 317/649] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:01 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:01.993429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:01.995713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:02.002118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:02.003389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:02.003831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:02.004523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:02.004979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:02.005407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:02.034496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ee7af3b3-71bf-4836-a7b4-a70d5e837bf2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045077 Sep 18 06:29:02.035074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 333/650] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:01 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:02.193755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:02.196000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:02.202323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:02.202916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:02.203375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:02.204087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:02.204537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:02.204960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:02.233201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e5467dd8-945a-4888-808e-fbf1291d7bc6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044163 Sep 18 06:29:02.233768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 318/651] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:02 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:02.259268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:02.259823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:02.265790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:02.266274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:02.266746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:02.267433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:02.267888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:02.268337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:02.369434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:02.370044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:02.375947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:02.376432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:02.378666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:02.379390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:02.379843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:02.380266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:02.397660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:02.406146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-97f1531d-a495-406c-b648-2991c46c9cac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150655 Sep 18 06:29:02.406832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 334/652] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:02.512847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:02.521397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:02.522863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:02.524013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-caa53d0f-c73f-49c0-800f-b2550ad79557 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158522 Sep 18 06:29:02.524569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 319/653] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:02 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:03.047247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:03.049426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:03.055831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:03.056314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:03.056751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:03.057432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:03.057887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:03.058310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:03.086524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e29a072-4c32-4d28-9ee9-5d0198cd1d71 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043263 Sep 18 06:29:03.087095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 335/654] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:03 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:03.245861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:03.248061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:03.255166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:03.255652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:03.256092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:03.256778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:03.257243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:03.257667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:03.288077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fbffdda1-2ac5-410d-ac83-cc95fbedea33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046124 Sep 18 06:29:03.288625 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 320/655] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:03 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:03.429218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:03.429821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:03.435761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:03.436239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:03.436670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:03.437347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:03.437796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:03.438214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:03.548998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:03.549640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:03.555603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:03.556082 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:03.556510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:03.557189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:03.557638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:03.558057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:03.566026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:03.568338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:03.576840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d682f533-34f3-4b45-a972-563940579eda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151392 Sep 18 06:29:03.577412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 336/656] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:03.684029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:03.693061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:03.694490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:03.695665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-048d3983-2e55-4f89-acaa-5ec804b52d54 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150633 Sep 18 06:29:03.696213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 321/657] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:03 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:04.099557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:04.101622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:04.108483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:04.108959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:04.109388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:04.110061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:04.110505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:04.110950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:04.139770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c76b4cc2-4f39-4242-90ff-d6b8a6839316 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044753 Sep 18 06:29:04.140314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 337/658] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:04 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:04.300871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:04.303857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:04.311739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:04.312227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:04.312663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:04.313356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:04.313811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:04.314238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:04.342883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f9f4837b-2eb1-47e2-a373-d5e081a50d75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046491 Sep 18 06:29:04.343439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 322/659] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:04 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:04.601265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:04.601902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:04.607767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:04.608254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:04.608686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:04.609369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:04.609822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:04.610247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:04.720297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:04.720942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:04.727402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:04.727934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:04.728322 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:04.729013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:04.729463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:04.729887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:04.741048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:04.749506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1f39e143-a04a-41fa-88f3-e88dbbfb7b19 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152105 Sep 18 06:29:04.750088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 338/660] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:04 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:04.857748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:04.866299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:04.867819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:04.868967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-55ca7ded-7894-4e41-aea2-554fd32cd808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152594 Sep 18 06:29:04.869521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 323/661] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:04 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:05.152134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:05.154357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:05.160926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:05.161403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:05.161839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:05.162515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:05.163087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:05.163509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:05.194093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e0342a13-2c4d-4885-9416-0b56fbbe46b0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045854 Sep 18 06:29:05.194670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 339/662] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:05 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:05.356880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:05.359038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:05.365928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:05.366405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:05.366867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:05.367566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:05.368013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:05.368428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:05.372131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:05.397760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4b67317f-d2dd-4ae3-89d8-4435c6185ee4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044771 Sep 18 06:29:05.398306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 324/663] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:05 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:05.773558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:05.774109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:05.780005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:05.780487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:05.780922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:05.781609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:05.782059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:05.782482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:05.895083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:05.895718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:05.901574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:05.902086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:05.902529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:05.903260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:05.903718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:05.904137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:05.913300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:05.921993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-424251ce-97e3-4c47-98e3-7a841df15c14 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152414 Sep 18 06:29:05.922566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 340/664] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:06.032545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:06.041920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:06.043383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:06.044533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7c18c4d9-5bcf-4bd4-98de-ca1f538687f1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153860 Sep 18 06:29:06.045092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 325/665] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:05 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:06.206989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:06.209132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:06.215400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:06.215868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:06.216295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:06.216966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:06.217408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:06.217825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:06.247035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bf3c0849-8dbe-4def-ab16-cf2d2cedc434 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044483 Sep 18 06:29:06.247564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 341/666] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:06 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:06.409255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:06.411443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:06.417767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:06.418247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:06.418719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:06.419399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:06.419855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:06.420277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:06.449199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9346c738-65b9-4ef0-98e7-b88933e5ed86 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043840 Sep 18 06:29:06.449744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 326/667] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:06 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:06.945771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:06.946377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:06.952333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:06.952815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:06.953250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:06.953933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:06.954386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:06.954837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:07.069484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:07.070093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:07.075990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:07.076471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:07.076907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:07.077634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:07.078035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:07.078455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:07.080060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:07.088296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-529f6438-66ce-4176-a1e1-3ab9aefb7172 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146421 Sep 18 06:29:07.088872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 342/668] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:07.207609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:07.216151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:07.217592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:07.218775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-094b3547-985e-4d38-bff3-9fe17dfbc6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153189 Sep 18 06:29:07.219327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 327/669] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:07 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:07.259541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:07.261715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:07.268081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:07.268563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:07.268997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:07.269677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:07.270130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:07.270554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:07.300481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-32432847-13b6-4468-844d-2e4fa899f815 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044907 Sep 18 06:29:07.301066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 343/670] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:07 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:07.461906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:07.463983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:07.470457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:07.470978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:07.472055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:07.472733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:07.473180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:07.473598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:07.505262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3ffff38d-ecd3-4daf-af20-8b3677957c03 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047323 Sep 18 06:29:07.505813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 328/671] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:07 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:08.112174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:08.113397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:08.119380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:08.119860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:08.120297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:08.120986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:08.121441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:08.121865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:08.244230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:08.244840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:08.248243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:08.250812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:08.251309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:08.251743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:08.252431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:08.252881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:08.253558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:08.256734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7c1928d0-3c7f-4a20-a952-90c1ea8410ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148416 Sep 18 06:29:08.257300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 344/672] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:08.312891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:08.315181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:08.322587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:08.323082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:08.323521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:08.324217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:08.324674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:08.325102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:08.364527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-39813504-ac82-4dff-89e2-337868b8949e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055454 Sep 18 06:29:08.365050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 345/673] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:08 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:08.385457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:08.394560 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:08.396424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:08.397174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dfc879f7-47cc-412c-a8de-9894ee85c803 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156846 Sep 18 06:29:08.397715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 329/674] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:08.517317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:08.520130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:08.526134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:08.526642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:08.527080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:08.527763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:08.528220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:08.528647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:08.557023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c26a5c78-d565-46d7-b029-31e52a18a22c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043682 Sep 18 06:29:08.557575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 346/675] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:08 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:09.280714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:09.281362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:09.287838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.288322 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.288760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.289444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.289899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.290323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.376929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:09.380341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:09.387587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.388155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.388694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.389380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.389828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.390246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.419641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:09.419813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9cef6990-47ab-4195-b9f6-8a0e002476e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046825 Sep 18 06:29:09.420319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 347/676] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:09 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:09.424709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:09.425296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:09.428399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b5b89ab3-f49f-4330-aaf0-bf69e639d21d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151497 Sep 18 06:29:09.428962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 330/677] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:09.431079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.431692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.432129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.432853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.433308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.433891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.563104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:09.569760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:09.572005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:09.572237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:09.573668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:09.574845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3a6487bf-f517-43f1-9365-80de5f860a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153604 Sep 18 06:29:09.575395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 348/678] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:09 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:09.577928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.578404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.578866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.579550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.579998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.580422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.609253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-35b094f6-064b-4d3c-97ce-cafc56e4a2a0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043907 Sep 18 06:29:09.609792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 331/679] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:09 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:09.851029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:09.853138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:09.859058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.859535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.859965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.860642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.861091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.861543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.890225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f54b9d4b-5240-4cd2-bac4-60b8f2c2b59d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043110 Sep 18 06:29:09.890791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 349/680] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:09 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 06:29:09.952254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:09.952754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:09.958646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.959134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.959572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.960257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:09.960709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:09.961133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:09.985466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Going to try to terminate instance {{(pid=120380) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:29:10.057494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0e90dea5-1065-4cf4-b463-77edaf9a48f6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 204 len: 0 microversion: 2.1 time: 0.108814 Sep 18 06:29:10.058236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 332/681] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:29:09 2022] DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:29:10.066939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:10.067483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:10.073291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.073763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.074195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.074894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.075340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.075756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.202598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:10.210942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-138c98a0-b2bf-4a2c-9bee-c4dc76af4992 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.148416 Sep 18 06:29:10.211500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 350/682] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:10 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:10.432530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:10.434767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:10.441758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.442237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.442703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.443384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.443838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.444259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.452192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:10.453006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:10.459583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.460150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.460693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.461960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.461960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.462426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.475005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-906cf0d7-290e-42b3-8ad1-43d8e8354913 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046679 Sep 18 06:29:10.475564 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 333/683] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:10 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:10.588845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:10.597225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d396e5eb-ea87-4bed-89db-bb3023bac25e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148926 Sep 18 06:29:10.597799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 351/684] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:29:10.599799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:10.600407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:10.606430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.607020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.607522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.608270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.608791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.609278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.622837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:10.625040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:10.631595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.632083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.632517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.633209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:10.633661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:10.634085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:10.663578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0ecdef0d-38ef-45c6-b11d-da8e9a42d34b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044506 Sep 18 06:29:10.664152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 352/685] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:10 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:10.742807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:10.751843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:10.752780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:10.754202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1d866e66-3f76-4a52-94fd-900f607263a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158311 Sep 18 06:29:10.754777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 334/686] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:11.234635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:11.235539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:11.241546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.242025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.242456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.243172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.243626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.244049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.259124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:11.369876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:11.378691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4c175cdd-c690-4349-9d62-fe505c93faca tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.148144 Sep 18 06:29:11.379275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 353/687] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:11 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:11.486900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:11.488962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:11.495432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.495906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.496337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.497012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.497459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.497876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.529023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6a80a25f-20d9-425b-9a79-6f2cd56023ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046146 Sep 18 06:29:11.529571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 335/688] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:11.622037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:11.622712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:11.630534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.631041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.631470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.632159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.632607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.633046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.676236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:11.678367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:11.684292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.684778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.685212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.685904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.686353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.686805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.717047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-63946d4b-f13d-40f4-b2c6-6fd27b08a2c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044811 Sep 18 06:29:11.717634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 336/689] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:11 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:11.766383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:11.775354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e988cf08-954f-4487-be5e-37f87765c473 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.157946 Sep 18 06:29:11.776065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 354/690] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:11.779067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:11.779676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:11.785474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.786069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.786506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.787210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:11.787659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:11.788090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:11.914803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:11.927040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:11.929740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:11.931687 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1193c49b-0766-437d-be81-71a2ecbfba11 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156413 Sep 18 06:29:11.932248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 337/691] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:11 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:12.401171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:12.401808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:12.407773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.408251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.408687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.409859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.410307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.410758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.538921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:12.541340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:12.543533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:12.546561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bd372999-01c8-4330-9e5e-08afc85b2a70 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.149206 Sep 18 06:29:12.547760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 355/692] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:12 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:12.550630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.551115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.551545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.552231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.552683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.553100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.580933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f4be70d7-5c51-4014-ba38-67d491473072 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043543 Sep 18 06:29:12.581471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 338/693] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:12 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:12.729464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:12.731558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:12.737941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.738449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.738911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.739589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.740036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.740453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.769603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b27b33d4-a6cc-4f77-8f3b-a88d1f03a02c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044203 Sep 18 06:29:12.770145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 356/694] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:12 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:12.797991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:12.798621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:12.804523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.805008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.805444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.806123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.806601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.807023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.810963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:12.938119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:12.947969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1f454c28-7301-47b6-ad72-3a0469a24fb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.153775 Sep 18 06:29:12.948531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 339/695] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:12.957594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:12.958190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:12.964055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.964531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.964964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:12.965638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:12.966084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:12.966507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:13.091671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:13.100150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:13.101596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:13.102784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-52a68486-d5a9-44a0-901a-ec10d15467cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149558 Sep 18 06:29:13.103341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 357/696] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:12 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:13.570330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:13.570976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:13.577476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:13.577958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:13.578394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:13.579111 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:13.579564 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:13.579986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:13.593156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:13.595598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:13.602137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:13.602651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:13.603090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:13.603777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:13.604360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:13.604791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:13.634073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4ade0499-0978-4210-8337-3215b4889941 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044832 Sep 18 06:29:13.634784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 358/697] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:13 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:13.709851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:13.717579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-05ea2501-305f-423c-80b3-7302738ff97b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.151081 Sep 18 06:29:13.718084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 340/698] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:13 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:13.781244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:13.783430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:13.789780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:13.790287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:13.790750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:13.791433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:13.791885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:13.792307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:13.821483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f09c3396-23f7-42de-94b3-e3e5bd12073c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044177 Sep 18 06:29:13.822029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 359/699] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:13 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:13.971521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:13.972052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:13.978042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:13.978528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:13.978999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:13.979684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:13.980139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:13.980566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.105588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:14.114210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-92bfe347-ed97-475f-a6e1-d7846e4fd034 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146647 Sep 18 06:29:14.114797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 341/700] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:13 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:14.126399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:14.127033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:14.133449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:14.133927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:14.134363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.135078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:14.135534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:14.135961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.330827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:14.339883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:14.341310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:14.342460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-df097a62-ea9c-45ad-be13-8c00744b5d5e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.219939 Sep 18 06:29:14.343089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 360/701] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:14 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:14.646934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:14.649192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:14.655710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:14.656189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:14.656622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.657302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:14.657749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:14.658175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.689211 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c5769a17-75a5-4bab-9342-050c48aea814 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046318 Sep 18 06:29:14.689764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 342/702] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:14.742165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:14.742867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:14.748913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:14.749387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:14.749817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.750652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:14.751101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:14.751515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.833797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:14.836022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:14.842309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:14.842833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:14.843275 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.844014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:14.844473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:14.844899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:14.877333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8cb26473-6db3-4e9c-9360-df4259939a4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047337 Sep 18 06:29:14.878160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 343/703] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:14 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:14.882219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:14.890081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a7a7597f-2fb8-4dec-a913-895afceca9cb tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.152007 Sep 18 06:29:14.890667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 361/704] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:14 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:15.138130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:15.138798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:15.140618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:15.146506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.147011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.147439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.148117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.148599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.149019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.326444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:15.335316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3c2ed39d-7709-4e11-ad39-5c41e657bd5d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.201147 Sep 18 06:29:15.335879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 344/705] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:15.372916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:15.373524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:15.375395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:15.381262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.381738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.382173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.382879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.383328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.383749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.515045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:15.524054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:15.525518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:15.526697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d00d8bbd-c5ad-4d5b-af8e-fc02ac09665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157670 Sep 18 06:29:15.527331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 362/706] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:15 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:15.701523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:15.703758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:15.711966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.712439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.712870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.713555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.714001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.714509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.743350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6e24cfb6-c245-4296-bac0-c45130bed9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045795 Sep 18 06:29:15.743895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 345/707] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:15 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:15.889381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:15.891536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:15.898436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.898958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.899396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.900088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.900540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.900968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.914085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:15.914723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:15.920640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.921196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.921629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.922307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:15.922788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:15.923212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:15.937947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5b5009fb-087e-429b-86ed-e03b774f343a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052497 Sep 18 06:29:15.938542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 363/708] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:15 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:16.050282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:16.057947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-858ab44a-1d63-45a5-8298-531121b3a644 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.147659 Sep 18 06:29:16.058508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 346/709] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:15 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:16.358527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:16.359157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:16.365600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:16.366077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:16.366510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:16.367327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:16.367778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:16.368261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:16.497399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:16.506298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dde08d31-14f1-4c12-898d-c0a722dd6d74 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151655 Sep 18 06:29:16.507124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 364/710] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:16.550815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:16.551422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:16.557331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:16.558079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:16.558231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:16.558939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:16.559389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:16.559809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:16.686160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:16.694700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:16.696142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:16.697288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1a5a408f-4e66-488e-87bc-d2e05aa3b87f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150392 Sep 18 06:29:16.697840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 347/711] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:16 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:16.755607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:16.757878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:16.764097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:16.764581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:16.765021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:16.765706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:16.766163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:16.766601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:16.796548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d4cd037b-aa19-4e35-bf97-8e3443e5ce2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045680 Sep 18 06:29:16.797100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 365/712] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:16.951094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:16.953344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:16.960398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:16.960884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:16.961322 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:16.962006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:16.962463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:16.962941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:16.991711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6aee0d75-9e7b-4af6-be46-64d5e784e076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044553 Sep 18 06:29:16.992254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 348/713] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:16 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:17.081720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:17.082327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:17.088728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:17.089206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:17.089641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:17.090320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:17.090803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:17.091231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:17.216508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:17.224060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a5f71579-522f-4dc7-bf45-98ec35b92d7c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.146745 Sep 18 06:29:17.224621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 366/714] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:17 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:17.530056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:17.530711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:17.536608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:17.537086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:17.537516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:17.538189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:17.538667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:17.539088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:17.668138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:17.676269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5969c568-447c-4b79-b016-9ec71643fb3e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150045 Sep 18 06:29:17.676823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 349/715] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:17.722468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:17.723032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:17.728972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:17.729450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:17.729878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:17.730561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:17.731029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:17.731444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:17.809259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:17.811471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:17.818327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:17.818841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:17.819277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:17.819971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:17.820419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:17.820841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:17.852708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9f2bca8c-261d-4331-8c37-497f09f5282f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047600 Sep 18 06:29:17.853283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 350/716] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:17 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:17.863918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:17.872688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:17.874134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:17.875359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ece0ca3d-3305-4239-902a-a123f51b5371 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156804 Sep 18 06:29:17.875910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 367/717] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:17 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:18.005063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:18.007230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:18.013654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.014150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.014570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.015273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.015724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.016149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.044358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2a2295bd-86fc-4fb6-928b-acdcab012bb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044261 Sep 18 06:29:18.044920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 351/718] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:17 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:18.247569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:18.248180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:18.254973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.255503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.255890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.256575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.257028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.257456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.385105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:18.392829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bfcd65fd-f391-4f3f-963a-66aa815ed03f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.149161 Sep 18 06:29:18.393387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 368/719] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:18 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:18.699149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:18.699793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:18.705642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.706652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.707089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.707793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.708243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.708663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.833925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:18.842093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-437d8c19-1159-486e-ba7c-5791163d75f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146852 Sep 18 06:29:18.842676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 352/720] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:18.865473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:18.867659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:18.873542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.874023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.874458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.875174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.875627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.876055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.900981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:18.901580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:18.904968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-95b37905-315f-489a-8853-21c076e0f6bd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043607 Sep 18 06:29:18.905522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 369/721] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:18 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:18.907404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.907879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.908309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:18.908988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:18.909991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:18.910413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.037408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:19.045837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:19.047331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:19.048475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-99ea8f73-8761-44dd-8cd8-0a516e64a930 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151288 Sep 18 06:29:19.049028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 353/722] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:18 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:19.056633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:19.058876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:19.064703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:19.065181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:19.066118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.066826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:19.067281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:19.067705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.071267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:19.097056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b9cc9e72-0b12-4ccf-be7c-9a5317c4965e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044379 Sep 18 06:29:19.097603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 370/723] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:19 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:19.417359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:19.418009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:19.424035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:19.424520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:19.425453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.426142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:19.426618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:19.427066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.552027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:19.559692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7d5df840-41f9-47af-92dc-f8f42a96424d tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.146187 Sep 18 06:29:19.560248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 354/724] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:19 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:19.865719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:19.866324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:19.872837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:19.873315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:19.873750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.874431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:19.874931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:19.875352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.918298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:19.920609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:19.927763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:19.928241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:19.928704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.929395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:19.929930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:19.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:19.955426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:19.960557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-37d4be05-24c5-4b91-ad8e-af6c9d5f6f69 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046458 Sep 18 06:29:19.961116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 355/725] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:19 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:20.073193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:20.073844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:20.079808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:20.080290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:20.080715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:20.081946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:20.082392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:20.082841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:20.093201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:20.101991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a2a4e162-32b1-4813-89fc-52610bd476c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.240669 Sep 18 06:29:20.102556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 371/726] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:20.119847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:20.122056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:20.128257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:20.128738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:20.129167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:20.129850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:20.130295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:20.130760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:20.162530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-abe20b43-fc41-4ee9-8a4f-9b01f456b001 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046214 Sep 18 06:29:20.163190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 372/727] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:20 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:20.213243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:20.222435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:20.223889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:20.225063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c5228ce6-f044-4706-bccc-112f880d0a5d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155722 Sep 18 06:29:20.225612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 356/728] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:20 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:20.583832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:20.584437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:20.590450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:20.590962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:20.591399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:20.592085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:20.592571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:20.592992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:20.720575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:20.728654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9f5f6d1c-fbcb-4497-8dce-3b3a8fc47853 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.149320 Sep 18 06:29:20.729216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 373/729] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:20 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:20.972430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:20.974486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:20.980912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:20.981397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:20.982179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:20.982556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:20.982966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:20.983393 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.020017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4e67e534-79a1-47d7-a537-917b30a13c84 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051571 Sep 18 06:29:21.020596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 357/730] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:20 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:21.125176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:21.125784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:21.132916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:21.133393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:21.133827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.134507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:21.134982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:21.135404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.175531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:21.177775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:21.183886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:21.184374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:21.184806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.185495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:21.185943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:21.186363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.220611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2afc3706-6a55-4f78-a731-e36b0252512d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048183 Sep 18 06:29:21.221973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 358/731] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:21 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:21.250279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:21.250951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:21.267796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:21.268302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:21.268742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.269450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:21.269902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:21.270480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.283074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:21.292408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f299a6b4-1710-498a-9144-09e9166a590c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.171040 Sep 18 06:29:21.293223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 374/732] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:21.442086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:21.451128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:21.452573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:21.454142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ea463124-f5b1-4ab5-a5c3-fbfa5aca90c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.208536 Sep 18 06:29:21.454722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 359/733] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:21 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:21.751472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:21.752080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:21.758375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:21.758887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:21.759317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.759995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:21.760449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:21.760871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:21.889697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:21.897857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4a10c0ca-74fd-4557-9de8-da1b94c2ba28 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.150193 Sep 18 06:29:21.898418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 375/734] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:21 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:22.032404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:22.034514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:22.041059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.041544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.041981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.042766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.043224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.043651 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.072288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c4736849-9227-4f88-931b-77c6783a7789 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043901 Sep 18 06:29:22.072829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 360/735] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:22 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:22.233759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:22.236131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:22.242508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.243018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.243451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.244139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.244592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.245015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.274897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-139a58bb-1778-4f94-b98c-1b8a565e320c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045323 Sep 18 06:29:22.275439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 376/736] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:22 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:22.317345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:22.317909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:22.323770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.324255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.324692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.325378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.325831 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.326256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.480090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:22.480709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:22.481993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:22.490032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.490522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.493775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.494736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.495208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.495639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.505339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-88c2072f-17ac-48f9-9325-7e76c3c42c59 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.192253 Sep 18 06:29:22.507822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 361/737] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:22 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:22.711888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:22.720727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:22.723931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:22.725132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-92d5ef01-148d-418f-a7e3-434ca205ce03 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.248875 Sep 18 06:29:22.725690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 377/738] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:22 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:22.921275 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:22.922103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:22.928031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.928516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.928947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:22.929639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:22.930087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:22.930506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.086928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:23.089425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:23.095822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.096318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.096760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.097460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.097924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.098386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.130001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8b9bf157-c1d0-4865-9a82-6371c1fbc43f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049050 Sep 18 06:29:23.130668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 378/739] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:23 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:23.134298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:23.141942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0db860b9-58d9-4dd3-8493-36b81244fd43 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1671 microversion: 2.1 time: 0.224504 Sep 18 06:29:23.142508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 362/740] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:22 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1671 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:23.287325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:23.289498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:23.295989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.296469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.296902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.297588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.298045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.298462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.328261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2491bab9-81c2-44c0-98fa-a9e7678cb953 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044942 Sep 18 06:29:23.328820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 379/741] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:23 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:23.463200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-a6de0454-f2f4-463d-a01c-bbe4393be437 req-6ce759ea-a28c-41c8-a480-b7001fbbd498 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "045c9108-b1cd-4720-94bc-7aa9159c26d4", "name": "network-vif-deleted", "tag": "9ebd90c7-97ce-45a5-8838-7c819266affb"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:23.473066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a6de0454-f2f4-463d-a01c-bbe4393be437 req-6ce759ea-a28c-41c8-a480-b7001fbbd498 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.473548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a6de0454-f2f4-463d-a01c-bbe4393be437 req-6ce759ea-a28c-41c8-a480-b7001fbbd498 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.473994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a6de0454-f2f4-463d-a01c-bbe4393be437 req-6ce759ea-a28c-41c8-a480-b7001fbbd498 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.486083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-a6de0454-f2f4-463d-a01c-bbe4393be437 req-6ce759ea-a28c-41c8-a480-b7001fbbd498 service nova] Creating event network-vif-deleted:9ebd90c7-97ce-45a5-8838-7c819266affb for instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 on hv-cinder-85535 Sep 18 06:29:23.491749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-a6de0454-f2f4-463d-a01c-bbe4393be437 req-6ce759ea-a28c-41c8-a480-b7001fbbd498 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032329 Sep 18 06:29:23.495382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 363/742] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:29:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:23.532198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:23.532793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:23.540196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.541180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.541789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.542695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.543297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.543873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.700711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:23.709162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f2cc83b0-0387-46f2-bd51-87819e775332 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.182549 Sep 18 06:29:23.709744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 380/743] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:23.755854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:23.756715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:23.769346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.771325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.772554 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.773298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:23.773776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:23.774213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:23.913679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:23.922443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:23.923912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:23.925106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-da4e44a3-1c91-4edc-99ca-0bd5abd22a06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.179235 Sep 18 06:29:23.925664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 364/744] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:23 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:24.141695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:24.144462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:24.150744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.151235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.151675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.152408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.152861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.153290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.165892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:24.166741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:24.174810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.175499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.175937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.176669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.177286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.177872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.187293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6e5b8535-e7cf-4343-b3a4-980304a60f3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049533 Sep 18 06:29:24.188106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 381/745] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:24 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:24.212419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.212896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.213335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.213806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lazy-loading 'fault' on Instance uuid 045c9108-b1cd-4720-94bc-7aa9159c26d4 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:29:24.255639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:24.263287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4aa8e236-400f-4fcb-ae85-0fb8adc7bdc6 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 200 len: 1327 microversion: 2.1 time: 0.101347 Sep 18 06:29:24.263873 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 365/746] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:24 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 1327 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:29:24.341162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:24.343351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:24.350188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.350707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.351149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.351844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.352296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.352719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.388292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7da91255-92f2-40bb-bc2f-b94414d5ba48 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050737 Sep 18 06:29:24.389373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 382/747] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:24 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:24.733288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:24.733897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:24.740329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.740813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.741269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.741953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.742400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.742866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.949870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:24.950479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:24.958095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.958773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.959318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:24.960175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:24.960767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:24.961477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.148947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:25.159428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5b09ca15-175c-44c9-bcf8-6ce7135d435a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1714 microversion: 2.1 time: 0.429415 Sep 18 06:29:25.163217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 366/748] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:24 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1714 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:25.199541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:25.202398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:25.208897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.209673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.210352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.211319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.211989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.212760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.237782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:25.248618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:25.250049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:25.250780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-84035e6d-e258-4a3d-af97-f9eca305a49d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054999 Sep 18 06:29:25.251235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5fc19b75-7544-4843-a6d4-ef79a85f774e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.305596 Sep 18 06:29:25.251466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 367/749] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:25 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:25.251777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 383/750] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:24 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:25.257075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:25.257630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:25.263436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.263994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.264505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.265265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.265793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.266299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.287663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:25.288717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:25.294841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.295389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.295889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.296825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.297530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.298269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.314121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:29:25.316090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:29:25.317605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:29:25.319316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:29:25.320785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.322897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:29:25.322897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b5412efb-444f-4029-9179-17430f6b8932 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.034793 Sep 18 06:29:25.322897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 384/751] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:25 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 06:29:25.326701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-833ce6ca-ae0d-4005-be3d-a7af77c77038 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '.version_select at 0x7fe67023e9d0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:25.348316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-833ce6ca-ae0d-4005-be3d-a7af77c77038 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1945423021" status: 202 len: 0 microversion: 2.1 time: 0.025161 Sep 18 06:29:25.349026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 385/752] 10.222.0.20 () {58 vars in 1221 bytes} [Sun Sep 18 06:29:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1945423021 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:29:25.361566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] HTTP exception thrown: Cannot 'detach_volume' instance a8254cee-e629-4d82-988c-708bac734c1e while it is in vm_state rescued Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Request method failure captured: Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments/6d3b844c-5d27-474e-a078-073931c04c5e HTTP/1.1 Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmU55Wg3gBiYn2aJSEvLyVJMV1h4CCP-z1W3AffyLY3fsW3JQ8OjuZv4iFhhTSFmKumF1s37EILY-hv50aLHoH2J1-083C3bmZIbw6-XE242ooMA_PlrxaMx2_DE94obNE1dW82mTeLq0C4XT8RYmdFLgmo0_0kmRhbQKjKMrBjrZ0 Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:29:25.363255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_91ef3478fd574180abe99393ba769124"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/91ef3478fd574180abe99393ba769124"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 7a6b8fbd31464e4d99449c74b4d1dda5 Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:29:25.364874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Cannot 'detach_volume' instance a8254cee-e629-4d82-988c-708bac734c1e while it is in vm_state rescued Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'a8254cee-e629-4d82-988c-708bac734c1e', 'id': '6d3b844c-5d27-474e-a078-073931c04c5e'} Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance a8254cee-e629-4d82-988c-708bac734c1e in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:29:25.365854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance a8254cee-e629-4d82-988c-708bac734c1e while it is in vm_state rescued Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance a8254cee-e629-4d82-988c-708bac734c1e while it is in vm_state rescued Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:29:25.367789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Returning 409 to user: Cannot 'detach_volume' instance a8254cee-e629-4d82-988c-708bac734c1e while it is in vm_state rescued {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:29:25.369283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6ade6ec4-070e-4c1a-b3cf-e10599618286 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments/6d3b844c-5d27-474e-a078-073931c04c5e" status: 409 len: 152 microversion: 2.1 time: 0.112094 Sep 18 06:29:25.369283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 368/753] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:29:25 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments/6d3b844c-5d27-474e-a078-073931c04c5e => generated 152 bytes in 114 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 18 06:29:25.374130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:25.374720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:25.381847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.382330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.382807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.383509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.383990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.384390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.403185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:25.405096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:25.414755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.415387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.415843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.416775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.417526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.418079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.454504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6629ff49-dfe5-47fc-b665-680babdad0d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057559 Sep 18 06:29:25.455374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 369/754] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:25 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:25.491569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-37e4fdba-7564-4130-898b-8fa531003eb1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/action" status: 202 len: 0 microversion: 2.1 time: 0.121068 Sep 18 06:29:25.495244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 386/755] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:29:25 2022] POST /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/action => generated 0 bytes in 123 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:29:25.501922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:25.503790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:25.511601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.512222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.512698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.513434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:25.513928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:25.514831 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:25.681488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:25.689733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d1a952e1-af06-4068-9995-3e4c454dcdfa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1722 microversion: 2.1 time: 0.193255 Sep 18 06:29:25.690299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 370/756] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1722 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:26.258687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-615716e2-8184-405b-a812-cafa6ac29aee tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'create', calling method: .version_select at 0x7fe6701fb160>, body: {"keypair": {"name": "tempest-keypair-615109679", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFFr7uxb0FOVsss3Frn/oNx8l6EGdakhpNBcmH0XLyWUOeSdc9sXdlsqukySTeQOAe8Djsbi+xzfxpNrOq7yAeFDcTunRaF7uIIUU9CspCpOo6AO26yJnzFfHur4NScYrQ=="}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:26.263703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:26.266496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:26.272705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:26.273410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:26.274143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-615716e2-8184-405b-a812-cafa6ac29aee tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Getting quotas for project 9cbc859294e9456eb2ca751279a17f52. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:29:26.274408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:26.275620 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:26.276349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:26.276780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:26.278338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-615716e2-8184-405b-a812-cafa6ac29aee tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Getting quotas for user 6c945a3e6a4946559aeadc63566d29f4 and project 9cbc859294e9456eb2ca751279a17f52. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:29:26.303158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-615716e2-8184-405b-a812-cafa6ac29aee tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.048858 Sep 18 06:29:26.304065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 387/757] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:29:26 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:26.308702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:26.309429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:26.317733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2fedb824-2ba8-4087-b7c7-2fdef27d952b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058020 Sep 18 06:29:26.317945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:26.318225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:26.318684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:26.319381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:26.319522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 371/758] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:26 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:26.320205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:26.320780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:26.340888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:26.468361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:26.473322 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:26.479644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:26.480212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:26.480712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:26.481469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:26.482175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:26.482698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:26.509369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:26.515641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-076d5831-ee99-411e-bac7-2c24aae1f73a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051533 Sep 18 06:29:26.518947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:26.520590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 372/759] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:26 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:26.527970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:26.529396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:26.530564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6cd0aa01-7f8b-47cb-aab3-8be183d52183 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.225582 Sep 18 06:29:26.531132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 388/760] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:26 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:26.712254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:26.713084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:26.719079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:26.719569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:26.720009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:26.720704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:26.721773 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:26.722208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:26.890685 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:26.899898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-771406c7-af95-4bb2-a0ba-7e8b3722fc85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1722 microversion: 2.1 time: 0.191626 Sep 18 06:29:26.900492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 373/761] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1722 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:27.331769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:27.334141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:27.340416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:27.340897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:27.341342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:27.342036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:27.342509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:27.342958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:27.372871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9e342a38-6075-4ca3-9130-e209ada81595 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045898 Sep 18 06:29:27.373434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 389/762] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:27 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:27.528194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:27.531176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:27.537514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:27.538065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:27.538504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:27.539222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:27.539675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:27.540099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:27.559703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:27.560487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:27.568618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:27.569365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:27.569886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:27.570733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:27.571311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:27.571822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:27.574145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4bbd7c02-0f38-4c3f-b27f-3b593c10e0a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049853 Sep 18 06:29:27.579185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 374/763] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:27 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:27.707632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:27.716234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:27.717748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:27.719049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-087893fd-91f8-40ec-9c36-b54aab3dccd5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.164320 Sep 18 06:29:27.719680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 390/764] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:27 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:27.758292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-597045163", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "65bb78e9-94da-42c7-9b2e-c8fc52cca325"}], "security_groups": [{"name": "tempest-securitygroup--391172048"}], "key_name": "tempest-keypair-615109679", "user_data": "}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:27.857917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] validate_networks() for [('65bb78e9-94da-42c7-9b2e-c8fc52cca325', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:29:27.923555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:27.924167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:27.930485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:27.930999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:27.931437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:27.932121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:27.932574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:27.932997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.109665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:28.120819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d8077273-0bad-4c01-b6e2-181626299b78 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1722 microversion: 2.1 time: 0.200984 Sep 18 06:29:28.121585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 391/765] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1722 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:29:28.326125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:29:28.326543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:29:28.327055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:29:28.327467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:29:28.327874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:29:28.336886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:28.337365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:28.337805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.338515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:28.338992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:28.339409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.355851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Getting quotas for project 9cbc859294e9456eb2ca751279a17f52. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:29:28.359673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Getting quotas for user 6c945a3e6a4946559aeadc63566d29f4 and project 9cbc859294e9456eb2ca751279a17f52. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:29:28.370094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:29:28.372384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:29:28.385968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:28.388176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:28.395369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:28.395859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:28.396299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.397041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:28.397463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:28.397920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.423419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-24d5ebd4-8284-47d9-9902-7362faf18562 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.669312 Sep 18 06:29:28.424066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 375/766] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:29:27 2022] POST /compute/v2.1/servers => generated 401 bytes in 672 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:29:28.431957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4d15f631-a615-4575-bf42-f87aedcb5b6f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049470 Sep 18 06:29:28.436838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 392/767] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:28 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:28.436976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-00f36bd2-0601-46a9-b6fe-b4e52700f44e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:28.436976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-00f36bd2-0601-46a9-b6fe-b4e52700f44e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:28.498427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-00f36bd2-0601-46a9-b6fe-b4e52700f44e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:28.505738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-00f36bd2-0601-46a9-b6fe-b4e52700f44e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1236 microversion: 2.1 time: 0.074374 Sep 18 06:29:28.506299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 376/768] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:28 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1236 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:29:28.586959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:28.589520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:28.597295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:28.597881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:28.598366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.599559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:28.600143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:28.600582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.635630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e13b8fa-bce7-4829-b3b8-653435f3efee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052832 Sep 18 06:29:28.636196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 393/769] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:28 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:28.746569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:28.747897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:28.757432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:28.758022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:28.758508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.759326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:28.759984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:28.760616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:28.906369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:28.916091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:28.917540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:28.918733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-703d6c0d-c08f-404b-b3fa-d3e766074cca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.177470 Sep 18 06:29:28.919284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 377/770] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:28 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:29.144792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:29.145633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:29.151876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.152581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.153016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.153806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.154286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.154745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.310511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:29.319127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6393ed82-c460-40e8-a8c3-efa0fc7a20bc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1728 microversion: 2.1 time: 0.178740 Sep 18 06:29:29.319981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 394/771] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1728 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:29.343552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:29.344275 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:29.354725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.355328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.355755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.356655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.357105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.358409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.441032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ce866904-b982-4af6-8684-35fda23d788b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.102015 Sep 18 06:29:29.441625 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 378/772] 10.222.0.20 () {58 vars in 1275 bytes} [Sun Sep 18 06:29:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-interface => generated 284 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:29.444594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:29.446751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:29.453279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.453772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.454332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.455047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.456025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.456449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.486657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-93623fc5-0a7f-488e-b840-15db8e04e69d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045893 Sep 18 06:29:29.487234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 395/773] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:29 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:29.528536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:29.529147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:29.535232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.535719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.536155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.536845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.537302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.537726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.624768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:29.634325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-32135ea2-d292-4323-8e34-821c3ca20a3a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1288 microversion: 2.1 time: 0.109602 Sep 18 06:29:29.646603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 379/774] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:29 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1288 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:29.649429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:29.652685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:29.660642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.661709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.662353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.663097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.664989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.665355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.702822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-876538e8-c8fc-4796-a7c0-16276d21967d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058277 Sep 18 06:29:29.703401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 396/775] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:29 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:29.945540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:29.946158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:29.952344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.953218 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.953847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:29.954557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:29.955332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:29.955519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:30.220326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:30.229909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:30.231530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:30.232936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8d2be2f7-99a6-4cb3-8727-2c92328a3dbf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.291931 Sep 18 06:29:30.233724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 380/776] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:29 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:30.504471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:30.522043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:30.537472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:30.541122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:30.547630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-61d852a5-1b99-4478-906b-24d10a46e68c req-7b3eaebf-4cc5-4291-aca3-4c56ebc91aac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a8254cee-e629-4d82-988c-708bac734c1e", "tag": "7ff7d3c9-95be-4eea-b0b3-a8bd5d358217"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:30.554416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:30.557032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:30.557635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:30.559149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:30.569474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-61d852a5-1b99-4478-906b-24d10a46e68c req-7b3eaebf-4cc5-4291-aca3-4c56ebc91aac service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:30.574842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-61d852a5-1b99-4478-906b-24d10a46e68c req-7b3eaebf-4cc5-4291-aca3-4c56ebc91aac service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:30.577653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-61d852a5-1b99-4478-906b-24d10a46e68c req-7b3eaebf-4cc5-4291-aca3-4c56ebc91aac service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:30.602125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-61d852a5-1b99-4478-906b-24d10a46e68c req-7b3eaebf-4cc5-4291-aca3-4c56ebc91aac service nova] Creating event network-changed:7ff7d3c9-95be-4eea-b0b3-a8bd5d358217 for instance a8254cee-e629-4d82-988c-708bac734c1e on hv-cinder-85535 Sep 18 06:29:30.607499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-61d852a5-1b99-4478-906b-24d10a46e68c req-7b3eaebf-4cc5-4291-aca3-4c56ebc91aac service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064044 Sep 18 06:29:30.608586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 381/777] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:29:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:30.623217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d8296399-ebc6-4d0e-8fd9-c5f8ee00c926 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.123188 Sep 18 06:29:30.624346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 397/778] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:30 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:30.666329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:30.667366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:30.673930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:30.674811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:30.675403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:30.676305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:30.676756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:30.677347 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:30.717502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:30.720375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:30.727304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:30.727810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:30.728253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:30.728965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:30.729421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:30.729845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:30.767062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6a4015ab-77fa-4593-9a65-36e34039cd91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053911 Sep 18 06:29:30.771745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 398/779] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:30 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:30.944128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:30.957025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b2afe600-6024-439c-bfc8-8723e89339ea tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1361 microversion: 2.1 time: 0.295334 Sep 18 06:29:30.958005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 382/780] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:30 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1361 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:31.262670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:31.264440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:31.274058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.275478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.276144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:31.277104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.277781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.278445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:31.463044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:31.475645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:31.477092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:31.478469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c9db7c64-36f9-40fe-89df-aca0f865de2e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.223045 Sep 18 06:29:31.479533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 399/781] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:31 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:31.492738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-13560084-f3e0-4920-b2ae-3914f3db52ba req-ef6cf551-e601-4e30-8e91-39593a406064 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08ce084d-f9c1-4025-a286-607e6b3407c3", "tag": "a5589a27-a50a-4b1d-b39f-fec92bc61176"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:31.509073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-13560084-f3e0-4920-b2ae-3914f3db52ba req-ef6cf551-e601-4e30-8e91-39593a406064 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.509815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-13560084-f3e0-4920-b2ae-3914f3db52ba req-ef6cf551-e601-4e30-8e91-39593a406064 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.510413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-13560084-f3e0-4920-b2ae-3914f3db52ba req-ef6cf551-e601-4e30-8e91-39593a406064 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:31.522451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-13560084-f3e0-4920-b2ae-3914f3db52ba req-ef6cf551-e601-4e30-8e91-39593a406064 service nova] Creating event network-changed:a5589a27-a50a-4b1d-b39f-fec92bc61176 for instance 08ce084d-f9c1-4025-a286-607e6b3407c3 on hv-cinder-85535 Sep 18 06:29:31.527255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-13560084-f3e0-4920-b2ae-3914f3db52ba req-ef6cf551-e601-4e30-8e91-39593a406064 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047743 Sep 18 06:29:31.528080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 383/782] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:29:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:31.636150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:31.639339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:31.646913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.647406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.647840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:31.648540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.649260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.649843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:31.682010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4c7b69bc-eb6a-49b6-a9ed-c37c3b90ceb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049871 Sep 18 06:29:31.682607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 400/783] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:31 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:31.784003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:31.786477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:31.788688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:31.794730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.795293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.795799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:31.796559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.797017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.797440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:31.828959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3fdcca00-1d5a-43a7-887e-05409d689b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050071 Sep 18 06:29:31.829526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 384/784] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:31 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:31.981061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:31.981670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:31.988326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.988808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.989238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:31.989917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:31.990366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:31.990826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:32.118393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:32.126018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7f209b5f-0a4d-4035-9d23-1145584d93ba tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1361 microversion: 2.1 time: 0.148836 Sep 18 06:29:32.126679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 401/785] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:31 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1361 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:32.507237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:32.507846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:32.515246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:32.515726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:32.516159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:32.516840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:32.517285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:32.518290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:32.628709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-23a6837b-7275-4ae7-9629-3a320b0268dd req-61b8d49e-af4e-4966-8254-e754d7333121 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a8254cee-e629-4d82-988c-708bac734c1e", "tag": "7ff7d3c9-95be-4eea-b0b3-a8bd5d358217"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:32.638003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-23a6837b-7275-4ae7-9629-3a320b0268dd req-61b8d49e-af4e-4966-8254-e754d7333121 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:32.638468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-23a6837b-7275-4ae7-9629-3a320b0268dd req-61b8d49e-af4e-4966-8254-e754d7333121 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:32.639199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-23a6837b-7275-4ae7-9629-3a320b0268dd req-61b8d49e-af4e-4966-8254-e754d7333121 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:32.650821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-23a6837b-7275-4ae7-9629-3a320b0268dd req-61b8d49e-af4e-4966-8254-e754d7333121 service nova] Creating event network-changed:7ff7d3c9-95be-4eea-b0b3-a8bd5d358217 for instance a8254cee-e629-4d82-988c-708bac734c1e on hv-cinder-85535 Sep 18 06:29:32.653451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:32.656050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-23a6837b-7275-4ae7-9629-3a320b0268dd req-61b8d49e-af4e-4966-8254-e754d7333121 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031724 Sep 18 06:29:32.656664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 402/786] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:29:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:32.662735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:32.664101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:32.665243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-620ea91a-b76b-436d-8a95-081cc763854a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.161909 Sep 18 06:29:32.665782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 385/787] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:32 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:32.695604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:32.697821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:32.703990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:32.704472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:32.704912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:32.705695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:32.706148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:32.706602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:32.735100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-237e9e75-74a2-40b9-a5ec-ffd4212925f0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043774 Sep 18 06:29:32.735646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 403/788] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:32 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:32.841277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:32.843528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:32.850357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:32.850870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:32.851302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:32.851974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:32.852422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:32.852838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:32.881112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b932b63a-dff3-4e13-b2a4-01919b1c0a39 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043761 Sep 18 06:29:32.881649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 386/789] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:32 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:33.149415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:33.150290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:33.156313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:33.156885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:33.157393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:33.158079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:33.159120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:33.159550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:33.637286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:33.644917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-36ee6db5-fec2-4c4d-ad3a-1cff2927e132 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1361 microversion: 2.1 time: 0.499454 Sep 18 06:29:33.645490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 404/790] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:33 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1361 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:33.689144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:33.690308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:33.696626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:33.697136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:33.697570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:33.698251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:33.698730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:33.699150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:33.748073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:33.750564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:33.757346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:33.757833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:33.758273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:33.759002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:33.759463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:33.759888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:33.785160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:33.790997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2f54d471-635b-4a84-b39c-beacad8a46cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046854 Sep 18 06:29:33.791610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 405/791] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:33 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:33.844114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:33.852765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:33.854170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:33.855337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bf964229-34b7-4fed-b2f6-985e49dca7fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.170063 Sep 18 06:29:33.855894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 387/792] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:33 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:33.893635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:33.895737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:33.897538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:33.902817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:33.903299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:33.903741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:33.904428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:33.904883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:33.905307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:33.936018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c16eba50-0e59-4c3a-acc5-b632278cfa0c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046686 Sep 18 06:29:33.936572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 406/793] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:33 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:34.667517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:34.668046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:34.673954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:34.674432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:34.675011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:34.675701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:34.676153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:34.676574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:34.803555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:34.804318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:34.805695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:34.811954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-de4f039a-5294-4550-9898-d9692ba7730e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1361 microversion: 2.1 time: 0.148304 Sep 18 06:29:34.812098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:34.812511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 388/794] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:34 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1361 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:34.812882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:34.813324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:34.814020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:34.815011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:34.815441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:34.845166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2ffc753f-7483-4882-80fd-44ac2e2a8088 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045445 Sep 18 06:29:34.846008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 407/795] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:34 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:34.881813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:34.882422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:34.888320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:34.888796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:34.889225 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:34.889904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:34.890350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:34.890798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:34.950035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:34.952530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:34.960447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:34.961147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:34.962160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:34.962898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:34.963705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:34.964137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:34.995222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d12b97e5-6097-41c5-86ac-bcdaba6ae116 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050016 Sep 18 06:29:34.995783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 408/796] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:34 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:35.026295 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:35.035501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:35.037102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:35.038273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dcbd5853-8d05-4ed7-87b7-3dd413eb7dea tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.161289 Sep 18 06:29:35.038865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 389/797] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:34 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:35.836035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:35.836591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:35.843355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:35.843828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:35.844260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:35.844938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:35.845382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:35.846469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:35.858038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:35.860202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:35.866701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:35.867187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:35.867618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:35.868303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:35.868756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:35.869177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:35.898585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-86ba530f-7993-4239-9e26-aac9e8127380 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044405 Sep 18 06:29:35.899150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 390/798] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:35 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:35.977868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:35.985475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1fcc58ca-8875-42f8-8228-17f916cffb1f tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1361 microversion: 2.1 time: 0.154117 Sep 18 06:29:35.986034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 409/799] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:35 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1361 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:36.006854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:36.009003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:36.014880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:36.015361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:36.015791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:36.016465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:36.016911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:36.017327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:36.046328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a01b2720-b4a5-49cd-acf5-99aa57baaa59 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043677 Sep 18 06:29:36.046905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 391/800] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:36 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:36.062989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:36.063605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:36.070040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:36.070519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:36.070990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:36.071674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:36.072127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:36.072550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:36.200171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:36.208642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:36.210112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:36.211314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2591acc0-7ebf-4cc2-967b-328b9cb914fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152170 Sep 18 06:29:36.211867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 410/801] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:36 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:36.911393 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:36.913502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:36.919907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:36.920383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:36.920813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:36.921488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:36.921933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:36.922352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:36.947611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:36.952969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2da10a5a-9a4a-4be8-a9c7-dd76911c5b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046197 Sep 18 06:29:36.953512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 392/802] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:36 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:37.008258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:37.008874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:37.010801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:37.016571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.017057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.018022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:37.018801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.019262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.019693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:37.059421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:37.061580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:37.068264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.068752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.069188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:37.069886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.070340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.070797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:37.108903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2fa744e6-e3a1-4626-a8f3-6a8fb2d4ebba tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048233 Sep 18 06:29:37.109352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 393/803] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:37 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:37.118534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-a813f467-6e2d-421a-b5db-89f77bed4545 req-27d0c96f-d546-471d-a929-192d4825c3a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08ce084d-f9c1-4025-a286-607e6b3407c3", "name": "network-vif-plugged", "status": "completed", "tag": "a5589a27-a50a-4b1d-b39f-fec92bc61176"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:37.128966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a813f467-6e2d-421a-b5db-89f77bed4545 req-27d0c96f-d546-471d-a929-192d4825c3a6 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.129454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a813f467-6e2d-421a-b5db-89f77bed4545 req-27d0c96f-d546-471d-a929-192d4825c3a6 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.129877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-a813f467-6e2d-421a-b5db-89f77bed4545 req-27d0c96f-d546-471d-a929-192d4825c3a6 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:37.140993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-a813f467-6e2d-421a-b5db-89f77bed4545 req-27d0c96f-d546-471d-a929-192d4825c3a6 service nova] Creating event network-vif-plugged:a5589a27-a50a-4b1d-b39f-fec92bc61176 for instance 08ce084d-f9c1-4025-a286-607e6b3407c3 on hv-cinder-85535 Sep 18 06:29:37.146466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-a813f467-6e2d-421a-b5db-89f77bed4545 req-27d0c96f-d546-471d-a929-192d4825c3a6 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031499 Sep 18 06:29:37.147813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 394/804] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:29:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:37.161305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:37.169799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-834d9b7c-76c6-4fbd-ab0f-804532868bb3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1361 microversion: 2.1 time: 0.165503 Sep 18 06:29:37.170378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 411/805] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:37 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1361 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:37.237163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:37.237887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:37.244359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.244869 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.245349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:37.246089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.246542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.247050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:37.432831 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:37.442129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:37.443633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:37.444790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4589d749-3235-4838-a6a2-efc354b44e06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.212364 Sep 18 06:29:37.445344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 395/806] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:37 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:37.966016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:37.968287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:37.974655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.975133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.975568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:37.976242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:37.976688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:37.977106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:38.006189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bfcabd9a-887d-4ffe-83e4-2b5a54227da3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044108 Sep 18 06:29:38.006761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 412/807] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:37 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:38.121418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:38.123617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:38.130257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:38.130780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:38.131222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:38.131906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:38.132360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:38.132787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:38.162109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-56c78305-a382-491d-9ac9-074d63932672 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044666 Sep 18 06:29:38.162662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 396/808] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:38 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:38.192833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:38.193446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:38.199808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:38.200293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:38.200729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:38.201420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:38.201878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:38.202306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:38.333982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:38.341473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-28505808-0a66-42e6-a97a-f9f82951ff22 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1361 microversion: 2.1 time: 0.152401 Sep 18 06:29:38.342029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 413/809] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:38 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1361 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:38.469470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:38.470086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:38.472579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:38.478183 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:38.478707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:38.479150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:38.479840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:38.480295 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:38.480721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:38.502542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=8 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:38.613796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:38.622639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:38.624065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:38.625215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-71638308-bd1a-42c1-921e-7dac2163752d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.159661 Sep 18 06:29:38.625762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 397/810] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:38 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:39.018806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:39.020912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:39.026869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.027346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.027778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.028452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.028900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.029320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.057981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ddeff4dd-fe02-467b-b8c5-0177a7791a34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043538 Sep 18 06:29:39.058511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 414/811] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:39 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:39.173588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:39.175760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:39.182933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.183575 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.184008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.184694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.185173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.185725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.223605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-17234dbb-bae1-4fba-b8ba-0655f88c3498 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053745 Sep 18 06:29:39.224196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 398/812] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:39 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:39.363176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:39.363779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:39.370505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.371024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.371466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.372154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.372609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.373035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.500059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:39.509186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-83ed7528-ceab-4962-abf3-a305fccd1410 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1546 microversion: 2.1 time: 0.149432 Sep 18 06:29:39.509839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 415/813] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:39 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1546 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:39.533216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:39.533708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:39.539838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.540355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.540790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.541504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.541953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.542376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.617118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2b88a3e9-589d-40f0-b56f-e402f03b6288 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.087797 Sep 18 06:29:39.617706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 399/814] 10.222.0.20 () {58 vars in 1275 bytes} [Sun Sep 18 06:29:39 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3/os-interface => generated 284 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:39.649760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:39.650374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:39.656352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.656839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.657278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.657971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:39.658433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:39.658887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:39.906465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:39.914937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:39.916368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:39.917519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-54951bae-6216-4c06-a942-618b7e5bcbb6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.271718 Sep 18 06:29:39.918073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 416/815] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:39 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:40.069600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:40.071946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:40.078137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:40.078751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:40.079191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:40.080069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:40.080069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:40.080069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:40.110696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-26b01bc2-2c46-4216-aeb8-5cc3953d84c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045174 Sep 18 06:29:40.111839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 400/816] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:40 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:40.237567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:40.239784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:40.245717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:40.246198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:40.246653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:40.247341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:40.247793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:40.248215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:40.278951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e35e88c0-9b0c-4376-bd2f-88930a8f025d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046541 Sep 18 06:29:40.279853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 417/817] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:40 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:40.675189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-60712395-fb15-4f11-959e-64099ac2bf89 req-29980924-4abb-4f68-a39e-8aec22f1ae68 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08ce084d-f9c1-4025-a286-607e6b3407c3", "tag": "a5589a27-a50a-4b1d-b39f-fec92bc61176"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:40.684616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-60712395-fb15-4f11-959e-64099ac2bf89 req-29980924-4abb-4f68-a39e-8aec22f1ae68 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:40.685205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-60712395-fb15-4f11-959e-64099ac2bf89 req-29980924-4abb-4f68-a39e-8aec22f1ae68 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:40.685700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-60712395-fb15-4f11-959e-64099ac2bf89 req-29980924-4abb-4f68-a39e-8aec22f1ae68 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:40.696524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-60712395-fb15-4f11-959e-64099ac2bf89 req-29980924-4abb-4f68-a39e-8aec22f1ae68 service nova] Creating event network-changed:a5589a27-a50a-4b1d-b39f-fec92bc61176 for instance 08ce084d-f9c1-4025-a286-607e6b3407c3 on hv-cinder-85535 Sep 18 06:29:40.701320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-60712395-fb15-4f11-959e-64099ac2bf89 req-29980924-4abb-4f68-a39e-8aec22f1ae68 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084443 Sep 18 06:29:40.701903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 401/818] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:29:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:40.943806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:40.944589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:40.950940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:40.951512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:40.952052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:40.952845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:40.953397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:40.953922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:41.124740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:41.128238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:41.134540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:41.135380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:41.136025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:41.137080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:41.137541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:41.137972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:41.168381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b13a89f6-d491-4f8d-88f9-829c11c2278b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048023 Sep 18 06:29:41.168975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 402/819] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:41 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:41.176973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:41.187277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:41.188735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:41.191107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bfd0dd68-5b82-413a-8d1a-81413d5e9e15 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.252123 Sep 18 06:29:41.191749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 418/820] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:40 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:41.292482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:41.295012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:41.303081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:41.303580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:41.304042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:41.304860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:41.305175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:41.305595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:41.342809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b1df7713-13c2-4cbb-9e60-b2bfd703d99f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054819 Sep 18 06:29:41.343368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 403/821] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:41 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:42.181164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:42.183308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:42.190799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:42.191282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:42.191714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:42.192461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:42.192958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:42.193385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:42.215556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:42.216199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:42.222765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-59b28d87-cab7-4246-a8ca-3957388a4190 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045458 Sep 18 06:29:42.222955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:42.223325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 419/822] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:42 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:42.223457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:42.223850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:42.224533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:42.224986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:42.225406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:42.358344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:42.367758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:42.369208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:42.370373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bc0f60cc-1b2b-4391-aa03-273266fe60d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158713 Sep 18 06:29:42.370966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 404/823] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:42 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:42.411642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:42.413826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:42.419749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:42.420223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:42.420654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:42.421323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:42.421771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:42.422188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:42.451263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-685dd9fe-4b07-4f10-a39e-387b2b57c0a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.099312 Sep 18 06:29:42.451748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 420/824] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:42 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:43.234915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:43.237199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:43.243711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:43.244202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:43.244634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:43.245317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:43.245768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:43.246189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:43.275396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f9fd27bb-9786-4375-bcc4-d2099888801d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044485 Sep 18 06:29:43.275946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 405/825] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:43 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:43.395263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:43.395863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:43.402667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:43.403148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:43.403583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:43.404306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:43.404758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:43.405182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:43.463578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:43.466092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:43.471875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:43.472357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:43.472787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:43.473481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:43.473933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:43.474356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:43.505110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9822502e-635c-4a2e-ab57-b760c194dd57 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045413 Sep 18 06:29:43.505684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 406/826] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:43 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:43.539212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:43.547736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:43.549701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:43.550892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-26984fbf-ede5-4c38-8bc0-2570fd5d969f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.159518 Sep 18 06:29:43.551446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 421/827] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:43 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:44.288518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:44.290677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:44.296999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:44.297475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:44.297903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:44.298622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:44.299073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:44.299494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:44.327618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f6a0ec73-eaac-498e-b523-36615ba02853 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043003 Sep 18 06:29:44.328169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 407/828] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:44 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:44.518114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:44.520362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:44.522685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:29:44.522836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:29:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments (ip 10.222.0.20) !!! Sep 18 06:29:44.529984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:44.530466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:44.530932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:44.531624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:44.532078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:44.532503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:44.546831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:44.562707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-db491a1d-bf08-4acf-a83f-067787429381 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049172 Sep 18 06:29:44.563268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 422/829] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:44 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:44.576263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:44.576902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:44.579403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:29:44.580121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:29:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 (ip 10.222.0.20) !!! Sep 18 06:29:44.586918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:44.587402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:44.587847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:44.588535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:44.588989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:44.589411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:44.715678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:44.724309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:44.725745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:44.726954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c4819d39-cdfd-4dd5-8288-2b8edc52c74b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154607 Sep 18 06:29:44.727506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 408/830] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:44 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:45.340433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:45.342567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:45.349180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:45.349670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:45.350109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:45.350830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:45.351293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:45.351722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:45.380664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c4364b47-86d0-489f-acae-2458c1844873 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044275 Sep 18 06:29:45.381210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 423/831] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:45 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:45.576171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:45.578336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:45.585194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:45.585669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:45.586100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:45.586807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:45.587258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:45.587672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:45.616677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bec6c518-84f3-4cba-b227-7a29b1d4ca15 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044445 Sep 18 06:29:45.617146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 409/832] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:45 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:45.752699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:45.753624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:45.760021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:45.760504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:45.760938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:45.761616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:45.762062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:45.762506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:45.897513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:45.906060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:45.907519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:45.909346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a9f95d26-a562-4c82-a933-4464f7b28f50 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.160212 Sep 18 06:29:45.909908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 424/833] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:45 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:46.393049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:46.395207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:46.401144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:46.401626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:46.402064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:46.402780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:46.403233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:46.403656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:46.432699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-572c0b7f-67b8-4dd4-99ca-ceb7db55523f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043610 Sep 18 06:29:46.433246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 410/834] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:46 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:46.629611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:46.631816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:46.637768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:46.638244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:46.638705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:46.639389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:46.639842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:46.640264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:46.668616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-444db9a0-16ea-4efd-98cd-63bdc4b34268 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043898 Sep 18 06:29:46.669160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 425/835] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:46 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:46.933844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:46.934378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:46.940801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:46.941284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:46.941722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:46.942403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:46.942889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:46.943315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:46.947230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:47.068156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:47.076634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:47.078071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:47.079258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-36483b0e-24bd-4fbc-b270-37ce5df82c0c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149478 Sep 18 06:29:47.079820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 411/836] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:46 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:47.445880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:47.448057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:47.454604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:47.455232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:47.455665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:47.456350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:47.456797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:47.457213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:47.487920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-63ca1763-31a4-495f-8f8c-5c32a9446b30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045898 Sep 18 06:29:47.488821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 426/837] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:47 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:47.681257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:47.683417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:47.689337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:47.689835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:47.690266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:47.691041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:47.691662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:47.692154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:47.722626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d40d7375-cd8d-4196-960d-b4dcd948d1c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045282 Sep 18 06:29:47.723174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 412/838] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:47 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:48.104669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:48.105208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:48.111219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:48.112228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:48.112669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:48.113363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:48.113820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:48.114250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:48.239921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:48.248487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:48.249939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:48.251134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-48017781-3009-4009-97ec-882bd401e1dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150447 Sep 18 06:29:48.251690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 427/839] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:48 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:48.500392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:48.502810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:48.509471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:48.510056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:48.510377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:48.511120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:48.511570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:48.511994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:48.541799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2828ae8c-7011-4023-ae20-306985ba7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045342 Sep 18 06:29:48.542352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 413/840] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:48 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:48.734087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:48.736198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:48.742618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:48.743100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:48.743533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:48.744208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:48.744654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:48.745090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:48.774372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-56cb3b52-3bc7-444e-868d-b7a06b258f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044238 Sep 18 06:29:48.774945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 428/841] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:48 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:49.276429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:49.277055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:49.283365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:49.283848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:49.284289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:49.284977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:49.285461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:49.285889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:49.427314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:49.436070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:49.437537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:49.438727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-601cb46d-d56b-4a6b-8456-caa3540ea924 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.166325 Sep 18 06:29:49.439285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 414/842] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:49 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:49.554992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:49.557176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:49.563107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:49.563590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:49.564023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:49.565361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:49.565846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:49.566272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:49.594153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f743e513-7c39-4443-af2c-49cfe83669b4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043249 Sep 18 06:29:49.594715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 429/843] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:49 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:49.787339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:49.789436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:49.795451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:49.795941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:49.796382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:49.797071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:49.797528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:49.797957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:49.826972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-82f8a494-1e08-4a0a-b802-5a57027d8599 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044216 Sep 18 06:29:49.827521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 415/844] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:49 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:50.464149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:50.464943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:50.471131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:50.471640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:50.472079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:50.472838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:50.473289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:50.473707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:50.603101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:50.606449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:50.608659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:50.611604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:50.613033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:50.614184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0a2bf12-d2b5-4658-b04e-cae325554561 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153933 Sep 18 06:29:50.614769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 430/845] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:50 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:50.614988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:50.615487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:50.615889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:50.616567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:50.617011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:50.617435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:50.646683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-16dbf124-4144-4b6f-a798-d0cb71b68a64 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044222 Sep 18 06:29:50.647229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 416/846] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:50 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:50.839463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:50.841596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:50.847627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:50.848105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:50.848540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:50.849223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:50.849673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:50.850094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:50.881234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-778974ee-37d9-4565-963c-cb9a66604700 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045755 Sep 18 06:29:50.881784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 431/847] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:50 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:51.639362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:51.639972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:51.646899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:51.647297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:51.647731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:51.648408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:51.648857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:51.649281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:51.659625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:51.661741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:51.668028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:51.668519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:51.668940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:51.669624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:51.670074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:51.670495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:51.699840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-87d53a9d-5b3e-4fcc-82c0-89649889a2a1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044569 Sep 18 06:29:51.700390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 432/848] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:51 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:51.782097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:51.791321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:51.792827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:51.793973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-06e8ba81-09fb-41fb-80e9-dd07a322c9dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158566 Sep 18 06:29:51.794518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 417/849] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:51 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:51.893933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:51.896084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:51.902005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:51.902486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:51.902958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:51.903640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:51.904090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:51.904542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:51.933383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-20bdd4f8-bba8-4177-8268-b18e1253a467 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044016 Sep 18 06:29:51.933925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 433/850] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:51 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:52.712283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:52.714333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:52.720631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:52.721598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:52.722024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:52.722722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:52.723168 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:52.723585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:52.752674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fcf9f376-3b71-4410-ae8b-8d3727474a95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044365 Sep 18 06:29:52.753200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 418/851] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:52 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:52.821146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:52.821750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:52.828316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:52.828799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:52.829234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:52.829919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:52.830421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:52.830887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:52.945555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:52.947731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:52.953542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:52.954044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:52.955154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:52.955903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:52.956367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:52.956801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:52.957096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:52.965624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:52.967023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:52.968190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0a9d8d8-3325-46a9-a85f-417d85290a31 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150892 Sep 18 06:29:52.968751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 434/852] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:52 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:52.986509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-656edf16-cd26-441d-b121-edd5cdd2fb98 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044879 Sep 18 06:29:52.987730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 419/853] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:52 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:53.764968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:53.767040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:53.773514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:53.773988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:53.774414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:53.775121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:53.775565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:53.775979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:53.804276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-723239c3-4d97-4e23-b568-6fffa8802a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043347 Sep 18 06:29:53.804822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 435/854] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:53 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:53.993609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:53.994218 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:53.999414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:54.000709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:54.001191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:54.001582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:54.001701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:54.003202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:54.003655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:54.004077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:54.008683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:54.009181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:54.009615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:54.010301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:54.010784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:54.011207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:54.040019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d04e2a99-a5af-40dd-a624-c53aa3f71527 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044366 Sep 18 06:29:54.040774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 436/855] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:53 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:54.128908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:54.137293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:54.138755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:54.139900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-209277cf-4817-4959-9b00-822dda85665a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150220 Sep 18 06:29:54.140450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 420/856] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:53 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:29:54.816592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:54.818803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:54.824752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:54.825232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:54.825667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:54.826374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:54.826849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:54.827278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:54.856348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-df6cdf10-305a-4612-8a2a-8ed21f658512 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043729 Sep 18 06:29:54.856890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 437/857] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:54 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:54.977664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:29:54.978085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:54.984125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:54.984585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:54.985014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:54.985694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:54.986142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:54.986561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.009097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lazy-loading 'flavor' on Instance uuid 4ce28d02-87b5-461d-bac3-2bf15b26fea9 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:29:55.051888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:55.054014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:55.060426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:55.060907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:55.061340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.062026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:55.062471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:55.062928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.081307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-11c9a1b9-abdf-468a-8acd-be8ce135e1e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "POST /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9/action" status: 202 len: 0 microversion: 2.1 time: 0.106609 Sep 18 06:29:55.082207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 421/858] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:29:54 2022] POST /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9/action => generated 0 bytes in 109 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:29:55.091179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:55.091878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:55.093906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f0e14869-2681-4e3c-8639-7f0f6aa1db0f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045834 Sep 18 06:29:55.094889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 438/859] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:55 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:55.102079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:55.102563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:55.103015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.103731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:55.104178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:55.104596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.164677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:55.165296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:55.171008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:55.171485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:55.171915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.172599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:55.173043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:55.173468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.177403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:55.278562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:55.286567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9e72bde9-0db7-4b74-bde5-1cdf53a317f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.199867 Sep 18 06:29:55.287234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 422/860] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:55 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:29:55.332620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:55.341072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:55.342500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:55.343676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4c8a2cc8-1a29-406c-9bd2-e6ee8ea0b1e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.182934 Sep 18 06:29:55.344405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 439/861] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:55 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:55.869239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:55.871417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:55.877630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:55.878116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:55.878552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.879294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:55.879749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:55.880174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:55.908256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-69cd63ef-12ba-43c8-bc5a-d03a91245fef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043498 Sep 18 06:29:55.909285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 423/862] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:55 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:56.106762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:56.108916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:56.115518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:56.116012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:56.116449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:56.117138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:56.117590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:56.118015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:56.147358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a52b4120-e72b-4036-9b48-5e8582d8f2b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044602 Sep 18 06:29:56.147906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 440/863] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:56 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:56.309145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:56.309677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:56.316087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:56.316571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:56.317005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:56.317688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:56.318143 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:56.318562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:56.368438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:56.369251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:56.375242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:56.375751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:56.376189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:56.377153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:56.377606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:56.378028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:56.492099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:56.499900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7b75ba3a-0835-4102-8c93-862f1c4c9a85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.194613 Sep 18 06:29:56.500500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 424/864] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:56 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:56.533772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:56.542373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:56.543861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:56.545020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a80115d7-c2ea-4416-a5f9-e1d772465b60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.180537 Sep 18 06:29:56.545577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 441/865] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:56 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:56.920187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:56.922358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:56.928728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:56.929214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:56.929650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:56.930337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:56.930819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:56.931244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:56.955269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:56.960386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b2f851c6-a224-46d9-8cb5-2428f725bb90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044166 Sep 18 06:29:56.960934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 425/866] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:56 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:57.160031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:57.162458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:57.168853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:57.169342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:57.169782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:57.170464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:57.170943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:57.171371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:57.200521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-de72fde7-102d-4e07-a4da-96e88ec8a58b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044932 Sep 18 06:29:57.201107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 442/867] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:57 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:57.522949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:57.523563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:57.531267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:57.531767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:57.532208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:57.532890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:57.533849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:57.534274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:57.569221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:57.569821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:57.575868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:57.576348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:57.576782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:57.577471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:57.577917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:57.578436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:57.716327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:57.724870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-784fbed1-3fa8-42b9-a260-dbb633dc43f4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.205870 Sep 18 06:29:57.725534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 426/868] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:57 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:57.745369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:57.754875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:57.756304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:57.757457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0236e207-9821-43ba-91eb-666a55f97b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.192067 Sep 18 06:29:57.760505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 443/869] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:57 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:57.973496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:57.975578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:57.981588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:57.982063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:57.982494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:57.983196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:57.983646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:57.984068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:58.012667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1a4be0b8-daa9-4a34-908a-0cccf700ccad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043772 Sep 18 06:29:58.013325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 427/870] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:57 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:58.214090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:58.216257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:58.223245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:58.223780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:58.224167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:58.224854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:58.225308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:58.225737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:58.256305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-55d6e0f8-7eb1-4a7e-ae3a-a12733d28cc5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047073 Sep 18 06:29:58.256857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 444/871] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:58 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:58.750159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:58.750880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:58.758051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:58.758542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:58.759009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:58.759699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:58.760155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:58.760592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:58.781348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:58.781947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:58.788538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:58.789021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:58.789453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:58.790139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:58.790601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:58.791033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:58.962838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:58.969388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:58.970497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c20428ed-fb5c-42aa-a550-2d67b061498b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.224176 Sep 18 06:29:58.971125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 428/872] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:58 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:29:58.978016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:58.979485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:29:58.980671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0366cf8c-221b-46e1-aa22-fbfa3a12095b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.203346 Sep 18 06:29:58.981215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 445/873] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:29:58 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:29:59.024858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:59.026947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:59.032724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:59.033208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:59.033645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:59.034332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:59.034810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:59.035240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:59.065620 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7dd1fc01-307b-4894-9646-8eac2c0cbc76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044590 Sep 18 06:29:59.066155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 429/874] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:59 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:29:59.268198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:59.270373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:29:59.276766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:59.277191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:59.277625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:59.278306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:29:59.278785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:29:59.279210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:29:59.308367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5a91769b-3af2-41c8-a4ad-0795ca748276 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044099 Sep 18 06:29:59.308922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 446/875] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:29:59 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:29:59.993163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:29:59.993793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:00.000524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:00.001034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:00.001464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:00.002404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:00.002882 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:00.003296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:00.005327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:00.005935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:00.011761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:00.012254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:00.012690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:00.013375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:00.013824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:00.014246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:00.194172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:00.198670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:00.203010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:00.204453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:00.205677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e4953046-81ee-403e-a063-8ddfef4e5195 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.204412 Sep 18 06:30:00.206453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 447/876] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:00 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:00.206597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-34a6ad80-38d2-4d52-a462-23366eebbe08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.217067 Sep 18 06:30:00.207073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 430/877] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:29:59 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:00.211068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:00.215572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:00.219223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:00.219708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:00.220148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:00.221350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:00.221803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:00.222230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:00.252363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-626de9de-177c-41cb-82ab-ab43c55455d8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044690 Sep 18 06:30:00.252920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 448/878] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:00 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:00.320638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:00.322792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:00.328619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:00.329106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:00.329540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:00.330331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:00.330814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:00.331241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:00.361099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8138d73a-ad27-4faf-8736-248d60802d5d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044266 Sep 18 06:30:00.362224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 431/879] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:00 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:01.228775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:01.229409 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:01.233082 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:01.233715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:01.235944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:01.236495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:01.236890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:01.237580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:01.238056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:01.238486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:01.240785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:01.241266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:01.241702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:01.242382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:01.242862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:01.243289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:01.438407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:01.445682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:01.447314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-927c8578-59d7-4fcf-b7c7-73e23f902035 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.222368 Sep 18 06:30:01.448173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 449/880] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:01 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:01.453449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:01.455598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:01.456850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:01.458431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:01.463418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-20e400de-597e-4fe3-969c-2a95bb8284cc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.230490 Sep 18 06:30:01.463418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 432/881] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:01 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:01.463664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:01.463664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:01.463664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:01.463664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:01.463962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:01.463962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:01.466913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:01.469136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:01.475377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:01.475948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:01.476459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:01.477213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:01.477731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:01.478392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:01.498383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a23b8491-70d0-49c8-bc5b-852309e18ee7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049222 Sep 18 06:30:01.498904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 450/882] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:01 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:01.508844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-edcc1905-6cb1-4a69-aa0f-15250fd1ac08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046917 Sep 18 06:30:01.509398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 433/883] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:01 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:02.477743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:02.478405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:02.483524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:02.484162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:02.484908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.485392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.485834 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.486485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:02.486643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.486997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.487435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.491690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.492176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.492644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.493371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.493822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.494245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.687128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:02.690864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:02.695572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ab3d6c7d-9136-4769-b0e4-b5bc291f027e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.221667 Sep 18 06:30:02.696454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 451/884] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:02 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:02.699824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:02.701342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:02.701473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:02.702542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-807a7a4d-385d-4fed-812b-e280b3ab318d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.222814 Sep 18 06:30:02.703329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 434/885] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:02 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:02.708615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:02.710705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:02.711563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:02.716615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.717092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.717414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.717521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.717890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.718186 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.718302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.718662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.719033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.719163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.719828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.720315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.747980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0b8f5791-6bfd-417a-8d75-a7c26bdc5a1c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043714 Sep 18 06:30:02.748546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 435/886] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:02 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:02.748697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bae8dd3e-ca2b-47c0-b348-4c590ebfe713 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050844 Sep 18 06:30:02.749124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 452/887] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:02 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:02.849308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:02.849808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:02.855792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.856276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.856712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.857397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:02.857850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:02.858273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:02.938921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lazy-loading 'services' on Instance uuid a8254cee-e629-4d82-988c-708bac734c1e {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:30:03.009038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c8082662-bb5b-40cd-b232-6b7cce42abc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments/6d3b844c-5d27-474e-a078-073931c04c5e" status: 202 len: 0 microversion: 2.1 time: 0.163547 Sep 18 06:30:03.009727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 436/888] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:30:02 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments/6d3b844c-5d27-474e-a078-073931c04c5e => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:30:03.018150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:03.020436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:03.026296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.026820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.027258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.027963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.028410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.028856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.059285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d1b41541-d0d5-40bc-bd78-d9d0bbc90f38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045153 Sep 18 06:30:03.059846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 453/889] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:03.719479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:03.720098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:03.726793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.727277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.727707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.727817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:03.728361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:03.728460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.728823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.729242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.734663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.735146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.735580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.736759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.737210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.737639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.924463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:03.928098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:03.932502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8dcb2cc2-1d91-4baf-b80c-e55dcba1fb22 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.217273 Sep 18 06:30:03.933156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 437/890] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:03 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:03.936762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:03.937674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:03.938191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:03.939372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f1354242-573b-49aa-a954-c81c670753be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215478 Sep 18 06:30:03.950031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:03.950258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 454/891] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:03 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:03.950258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:03.950461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.950461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.951123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:03.951229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.951229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.951229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.951229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.951510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=98 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:03.953870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.954353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.954817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.955499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:03.957337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:03.957337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:03.979845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cc4889be-1508-452d-b5ee-2cb3c63debe9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045655 Sep 18 06:30:03.980524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 438/892] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:03 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:03.983448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:03.989172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-714ebdd3-77e0-4c25-a4f7-745017603b08 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048336 Sep 18 06:30:03.989726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 455/893] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:03 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:04.071558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:04.073663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:04.079482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:04.079961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:04.080391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:04.081066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:04.081514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:04.081933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:04.110585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-afe6162e-5a0e-4dda-9342-84bf54f558ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042871 Sep 18 06:30:04.111809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 439/894] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:04 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:04.956197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:04.956811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:04.963559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:04.963979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:04.964414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:04.964526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:04.964608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:04.965146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:04.965600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:04.966026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:04.971240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:04.971727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:04.972156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:04.972842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:04.973323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:04.973745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:05.168073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:05.174766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:05.175735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b8f4bc40-9075-48bb-9b70-d68091998f2f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.223369 Sep 18 06:30:05.176306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 456/895] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:04 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:05.180854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:05.183049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:05.183528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:05.184980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:05.186151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bbdc3a36-8d4a-481f-8bc2-d4d3f7c9b54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.226248 Sep 18 06:30:05.186860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 440/896] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:04 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:05.191173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:05.193362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:05.194835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:05.195508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:05.195953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:05.196652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:05.197162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:05.197587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:05.199226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:05.199712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:05.200152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:05.200844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:05.201632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:05.201768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:05.228322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-85e401b8-366c-4b3e-a60d-bf73b9e2b06a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050906 Sep 18 06:30:05.229242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 457/897] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:05 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:05.234161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:05.236752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:05.240031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:05.243127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:05.243688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:05.244212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:05.244942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:05.245443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:05.245869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:05.246594 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fd6a0ec8-6187-4eb1-b8c8-e05ab17d2ead tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058902 Sep 18 06:30:05.247177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 441/898] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:05 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:05.275721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b26b0c0c-2a3c-4561-89fb-d266c7737cdb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045233 Sep 18 06:30:05.276288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 458/899] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:06.199570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:06.200174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:06.211806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:06.212386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.212507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:06.212775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.213209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.213908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.214365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.214807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.219073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.219559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.220018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.221637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.221637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.221637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.405181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:06.408345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:06.413237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1cf63a39-d55c-4c54-ad8a-d08c7411db3d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.217895 Sep 18 06:30:06.413862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 442/900] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:06 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:06.417053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:06.418259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:06.418508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:06.419708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc6f3249-1397-49b8-a7be-a75ecb526a09 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.211801 Sep 18 06:30:06.428796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 459/901] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:06 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:06.428993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:06.429086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:06.429178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.429178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.429178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.429178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.429484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:06.429572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.429572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.435539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.436033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.436478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.437181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.437639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.438067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.463740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-40b1d8f6-c6ec-48cc-964d-ca2263b5d950 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048906 Sep 18 06:30:06.464654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 443/902] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:06 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:06.468932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-277b654f-0e76-4380-90f9-1b27ea6b111e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047651 Sep 18 06:30:06.469502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 460/903] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:06 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:06.470051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:06.472236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:06.477950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.478423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.478895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.479579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:06.480029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:06.480449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:06.509584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cd2eda92-9e42-4c31-bcc6-6808268f7d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043724 Sep 18 06:30:06.510127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 444/904] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:07.436237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:07.436847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:07.443899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.444382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.444814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.445497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.445948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.446368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.447589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:07.448265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:07.455013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.455511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.455955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.456656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.457110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.457534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.645107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:07.649239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:07.652959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e5fef39c-6120-42a2-9919-ab278f55d76a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.220506 Sep 18 06:30:07.653535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 461/905] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:07 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:07.658500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:07.658663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:07.659981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:07.660702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:07.661152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-90e85949-316b-45e5-8865-39b78d7b7d08 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.217379 Sep 18 06:30:07.661855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 445/906] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:07 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:07.666302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:07.668522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:07.674570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.675020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.675336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.675562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.676012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.676134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.676775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.676892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.677252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.677360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.677704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.677811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.706796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-656c8a74-defa-459f-8bb7-afb136b7819d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052130 Sep 18 06:30:07.707354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 462/907] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:07 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:07.708383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-10cef882-4647-4004-9eb2-a0500467f1f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045345 Sep 18 06:30:07.709274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 446/908] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:07 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:07.711837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:07.713918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:07.719498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.719972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.720902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.721576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:07.722021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:07.722438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:07.751054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-66c4f7ae-ef53-4efc-a6e5-7cc258bfae71 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042655 Sep 18 06:30:07.751903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 463/909] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:08.677568 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:08.678172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:08.684709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:08.685138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:08.685571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:08.686249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:08.686721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:08.687143 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:08.687407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:08.688128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:08.694472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:08.694981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:08.695417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:08.696108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:08.696557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:08.696980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:09.126986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:09.133615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:09.134811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2d1e50d7-260f-40ca-af3d-a1129d01c4be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.461740 Sep 18 06:30:09.135379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 447/910] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:08 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:09.139958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:09.142671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:09.148146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:09.148399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:09.148879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:09.149309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:09.149905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:09.150024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:09.150425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:09.150886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:09.151266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0502be81-24d9-4261-8eee-b1882675aadf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.467750 Sep 18 06:30:09.151823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 464/911] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:09.156359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:09.159078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:09.165134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:09.165390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:09.165827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:09.166515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:09.166989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:09.167412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:09.180507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9aac464d-cadc-49e8-a077-2ec0de9ff77f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044112 Sep 18 06:30:09.181066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 448/912] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:09 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:09.185626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:09.187739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:09.193732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:09.194209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:09.194703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:09.195384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:09.195832 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:09.196250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:09.197135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bf402f08-0e7c-47c9-9efd-7da953a537ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044281 Sep 18 06:30:09.197696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 465/913] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:09 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:09.226054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-24b1a003-d248-4c8f-bc39-0fc17d4f19db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043986 Sep 18 06:30:09.226718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 449/914] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:10.158882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:10.159566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:10.166033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.166515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.166983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.167668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.168123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.168549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.179635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:10.180270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:10.186687 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.187181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.187624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.188304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.188755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.189175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.358010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:10.365766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ec2fcf1a-3396-4af0-846d-d2b7b804910f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.211450 Sep 18 06:30:10.366552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 466/915] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:10 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:10.370931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:10.373052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:10.378859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:10.379177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.379661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.380096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.380789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.381236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.381659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.387552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:10.389014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:10.390162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-335acb99-c76d-451e-813e-f73c8fc69c4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214387 Sep 18 06:30:10.390733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 450/916] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:10.396069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:10.398091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:10.403804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.404301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.404723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.405429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.405876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.406289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.412348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6a959a53-b5c2-42f5-b84b-c4ec85ba6d98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044846 Sep 18 06:30:10.412842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 467/917] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:10 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:10.417365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:10.419480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:10.425057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.425536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.425972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.426754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:10.427209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:10.427634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:10.435990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-75d9fab8-1461-4099-8c1b-04393efbddfc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044143 Sep 18 06:30:10.436897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 451/918] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:10 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:10.457109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-af115a18-b39a-42c7-aa6f-8e8e27b56a08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043236 Sep 18 06:30:10.457671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 468/919] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:11.339419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-bdd98346-0cb7-4a46-a62e-35ac177c3b14 req-ae43b293-8248-4c7a-8cc7-6dcf315d0178 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ce28d02-87b5-461d-bac3-2bf15b26fea9", "name": "network-vif-unplugged", "status": "completed", "tag": "726ec9db-ceda-4200-b057-55e60adeb8a9"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:30:11.349311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-bdd98346-0cb7-4a46-a62e-35ac177c3b14 req-ae43b293-8248-4c7a-8cc7-6dcf315d0178 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.349790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-bdd98346-0cb7-4a46-a62e-35ac177c3b14 req-ae43b293-8248-4c7a-8cc7-6dcf315d0178 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.350227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-bdd98346-0cb7-4a46-a62e-35ac177c3b14 req-ae43b293-8248-4c7a-8cc7-6dcf315d0178 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.361393 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-bdd98346-0cb7-4a46-a62e-35ac177c3b14 req-ae43b293-8248-4c7a-8cc7-6dcf315d0178 service nova] Creating event network-vif-unplugged:726ec9db-ceda-4200-b057-55e60adeb8a9 for instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 on hv-cinder-85535 Sep 18 06:30:11.366064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-bdd98346-0cb7-4a46-a62e-35ac177c3b14 req-ae43b293-8248-4c7a-8cc7-6dcf315d0178 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.086524 Sep 18 06:30:11.366666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 452/920] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:30:11 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:11.389049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:11.389691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:11.396833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.397340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.397779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.398508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.398997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.399453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.417061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:11.417781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:11.423814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.424294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.424723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.425407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.425851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.426269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.583048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:11.590773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-87f5593f-2a0b-42b4-976e-58618d710501 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.205516 Sep 18 06:30:11.591353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 469/921] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:11 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:11.595884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:11.598129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:11.604470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.604967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.605402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.606316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.606828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.607259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.610299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:11.620457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:11.622445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:11.624046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-487b99bc-01ed-44f6-89cc-8b87b9eae194 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.211133 Sep 18 06:30:11.635410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 453/922] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:11 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:11.650984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:11.651650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-844131dc-9358-4f4c-91d3-d33d1b43eafb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059252 Sep 18 06:30:11.656588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 470/923] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:11.656813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:11.657447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:11.659632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:11.660624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.661212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.661744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.662546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.663112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.663633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.665591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.666128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.666503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.667226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:11.667911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:11.668105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:11.693909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-53a70b53-f4c3-4642-89ff-063c5327cbf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057297 Sep 18 06:30:11.694834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 454/924] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:11 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:11.697807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f5cc3732-f41a-4e1f-a03b-d9b1c2b026e0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044487 Sep 18 06:30:11.698420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 471/925] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:12.657677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:12.658281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:12.664023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:12.664950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.665432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.665866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.666202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:12.666544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.667022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.667432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.672716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.673692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.674135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.674836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.675286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.675707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.689791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:12.706478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8e16f3b7-30c0-4594-8959-5c7ecb083532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046285 Sep 18 06:30:12.707089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 472/926] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:12 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:12.711611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:12.712208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:12.717944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.718418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.718880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.719560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.720004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.720421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.847240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:12.855936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:12.857369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:12.858539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c7b71a01-eb74-4f57-b672-8e1f970bde58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.204790 Sep 18 06:30:12.859429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 455/927] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:12 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:12.864407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:12.866514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:12.872463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.872949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.873390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.874076 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.874530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.874992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.879685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:12.887521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2a806fc1-121f-4205-bf5b-1c7f502b976d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.179488 Sep 18 06:30:12.888088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 473/928] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:12 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:12.893288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:12.895501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:12.901296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.901759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.902197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.902898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:12.903346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:12.903760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:12.905897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-96e33fc0-6670-40cd-b394-63d91e7841c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045441 Sep 18 06:30:12.906458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 456/929] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:12 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:12.933097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c24ed839-38ff-4fc2-8913-a4deff019d26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043951 Sep 18 06:30:12.933642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 474/930] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:13.718948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:13.721024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:13.727494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:13.727978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:13.728413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:13.729098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:13.729556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:13.729982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:13.759251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e6ba9b15-65fa-4e2b-942e-08a36cd53217 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044462 Sep 18 06:30:13.759801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 457/931] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:13 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:13.885034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:13.885637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:13.892623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:13.893101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:13.893532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:13.894205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:13.894680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:13.895098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:13.910443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:13.911088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:13.917052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:13.917534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:13.918001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:13.918709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:13.919164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:13.919587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:14.091606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:14.099978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:14.100596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:14.102071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:14.103265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-73d00dd7-fd01-4b1b-b349-eb1e856d0d62 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.222202 Sep 18 06:30:14.114648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 475/932] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:13 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:14.114854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ffaa4e51-50a9-44d7-a32d-811b35b649dc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.201159 Sep 18 06:30:14.114962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:14.115058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 458/933] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:13 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:14.115159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:14.115258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:14.118606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:14.118729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:14.118729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:14.118729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:14.118729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:14.119033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:14.121641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:14.122368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:14.122972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:14.123505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:14.124286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:14.124815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:14.125239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:14.129573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:14.149888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6ddaaa54-91b6-493f-85cd-9d1ddbdaee0b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045041 Sep 18 06:30:14.150455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 476/934] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:14 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:14.155710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-70f4efc2-a22a-43e0-bddb-552c6c329379 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046372 Sep 18 06:30:14.156336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 459/935] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:14.771822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:14.773981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:14.779977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:14.780456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:14.780885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:14.781560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:14.782007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:14.782951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:14.811477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b368826c-6a8a-4b5c-8ba4-b22f6a4b9e37 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043639 Sep 18 06:30:14.812019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 477/936] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:15.130649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:15.131256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:15.132442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:15.133041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:15.137824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.138376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.138843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.139353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.139535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.139825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.139991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.140252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.140444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.140930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.141380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.142301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.348403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:15.353977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:15.357562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:15.359038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:15.360256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-47e1d000-a1aa-417d-9897-1d2a310548cd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.233933 Sep 18 06:30:15.360804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 460/937] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:15 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:15.362648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-06612aa5-a7c4-427d-80f4-99b37e6243d3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.233932 Sep 18 06:30:15.363239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 478/938] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:15 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:15.365420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:15.367797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:15.368616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:15.369887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:15.379305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.379495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.379595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.379691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.379822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.379918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.380003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.380080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.380177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.380279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.380378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.380459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.410638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0fdf7ff1-9e2b-41d7-a608-47ea948dccb4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046351 Sep 18 06:30:15.411099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e969cd20-7f4e-47f6-aa28-b7a56bfce88c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049240 Sep 18 06:30:15.411215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 479/939] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:15.411879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 461/940] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:15 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:15.823996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:15.826076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:15.832066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.832577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.833010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.833695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:15.834147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:15.834570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:15.864371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-352434c6-9db8-4176-8f2e-16cfcbc643ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044374 Sep 18 06:30:15.864934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 480/941] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:15 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:16.386979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:16.387539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:16.394464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.394973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.395407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.396089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.396535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.396953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.398924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:16.399538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:16.406085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.406565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.407038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.408362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.408815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.409239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.594128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:16.597776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:16.602898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:16.604350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:16.605508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9eaa36d2-c3fa-4815-950d-bf4837ba86ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.222581 Sep 18 06:30:16.605710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-db662fc2-26a1-4f6a-90cf-2e27246c587b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.210688 Sep 18 06:30:16.606066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 462/942] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:16 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:16.618819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 481/943] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:16 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:16.619026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:16.619124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:16.619212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:16.619307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:16.619392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.619646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.620027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.620649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.620776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.621139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.621261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.621577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.621700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.625068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.625068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.625068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.660095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cf245c0a-0cb7-4580-a05c-bc0ccf1ff08c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052234 Sep 18 06:30:16.660673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 482/944] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:16 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:16.664015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7cdb384f-bfe0-4dbc-95d4-0c3d3b5b3b43 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056912 Sep 18 06:30:16.665074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 463/945] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:16.804456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:16.804948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:16.810860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.811341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.811779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.812463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.812915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.813333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.836056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9bb7f1c5-5478-4833-9439-1e02e33556b7 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3/ips" status: 200 len: 144 microversion: 2.1 time: 0.082852 Sep 18 06:30:16.836604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 483/946] 10.222.0.20 () {58 vars in 1248 bytes} [Sun Sep 18 06:30:16 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3/ips => generated 144 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:16.877129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:16.879290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:16.885162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.885639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.886070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.886790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:16.887240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:16.887657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:16.917138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-628fa836-fb2f-48f0-8358-2a6285d44f7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044159 Sep 18 06:30:16.917608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 464/947] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:17.630835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:17.631438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:17.632837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:17.633520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:17.637931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.638408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.638864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.639547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.639726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.639995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.640205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.640411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.640635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.641317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.641766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.642186 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.785087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:17.794219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:17.795702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:17.796879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d2ceb414-f9ed-4049-b062-86b2052563ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.170332 Sep 18 06:30:17.797443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 484/948] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:17 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:17.801922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:17.804208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:17.810847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.811333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.811774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.812470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.812929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.813364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.843983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d21318a4-e447-4cc5-a544-4260e02f1678 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045489 Sep 18 06:30:17.844555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 485/949] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:17 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:17.849121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:17.851454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:17.857213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.858335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.858335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.858861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.859323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.859753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.891901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-df12903a-c30f-4b03-8d8b-8e39c56f29df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046218 Sep 18 06:30:17.892648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 486/950] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:17.924967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:17.929736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:17.931993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:17.932781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dfd6921b-bfc5-4f05-9124-2cc853432636 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.303790 Sep 18 06:30:17.933345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 465/951] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:17 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:17.937852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.938339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.938823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.939585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:17.940560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:17.940989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:17.970062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-351613f5-2c98-46de-87d9-fa48260f88af tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044162 Sep 18 06:30:17.970648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 487/952] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:17 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:18.822016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:18.822595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:18.828715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.829205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.829648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.830384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.830931 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.831362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.856606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:18.858804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:18.865404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.865894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.866346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.867052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.867509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.867935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.898480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-30aecd83-d791-4521-9f4d-bbd25a5a4608 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045628 Sep 18 06:30:18.898802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 488/953] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:18 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:18.903318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:18.905438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:18.911767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.912252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.912688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.913380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.913837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.914261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.928538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:18.945569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-18e3ffb7-ae85-41a0-9a51-f2e848694cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045742 Sep 18 06:30:18.946150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 489/954] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:18.955892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:18.956513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:18.962437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.962960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.963399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.963849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:18.964090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.964552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.964975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.972879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:18.974297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:18.975480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2a82be26-5226-4208-9f19-c540ca3b3e3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157637 Sep 18 06:30:18.976029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 466/955] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:18 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:18.982354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:18.984701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:18.990801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.991284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.991716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:18.992624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:18.993137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:18.993563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:19.022376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-59384c1b-39bb-4330-a1a3-d18252ef7e13 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043479 Sep 18 06:30:19.023137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 467/956] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:18 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:19.095457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:19.103571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3af13fb7-6c25-495b-9c20-368aa8676fc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151459 Sep 18 06:30:19.104152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 490/957] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:18 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:19.910309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:19.912490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:19.919223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:19.919730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:19.920152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:19.920838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:19.921292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:19.921726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:19.950781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-479eff1b-409e-4545-8c44-6676d3d04bca tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044525 Sep 18 06:30:19.951353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 468/958] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:19 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:19.959400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:19.961516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:19.967904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:19.968394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:19.968823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:19.969494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:19.969940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:19.970356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:19.998553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ff116f2a-8eca-4e46-ab55-1ff3702c8a31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043768 Sep 18 06:30:19.999125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 491/959] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:20.000967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:20.002771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:20.008560 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:20.009043 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:20.009480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:20.010164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:20.010637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:20.011066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:20.034601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:20.036709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:20.042479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:20.043008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:20.043423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:20.044102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:20.044549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:20.044969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:20.074717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a1e8a90a-5c94-48e7-9089-f7479cda598c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043907 Sep 18 06:30:20.075279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 492/960] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:20 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:20.128537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:20.129147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:20.135106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:20.135592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:20.136146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:20.136694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:20.136830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:20.137285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:20.137709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:20.145254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:20.146712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:20.147872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-901d0040-2328-4e40-926c-2bc2cd54de75 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150753 Sep 18 06:30:20.148415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 469/961] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:19 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:20.268842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:20.276509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d3677ec7-17ea-46af-a958-8e9ef336a75c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152797 Sep 18 06:30:20.277076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 493/962] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:20 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:20.964151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:20.966228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:20.972898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:20.973467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:20.973910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:20.974631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:20.975121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:20.975552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.007073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0fd63420-5a8f-4660-9cde-2a1365501aeb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046875 Sep 18 06:30:21.007656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 470/963] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:20 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:21.011785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:21.014019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:21.020691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:21.021181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:21.021611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.022304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:21.022816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:21.023247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.053167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a7d3f0c0-169d-4ef8-9425-d832fc7da7fc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045393 Sep 18 06:30:21.053765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 494/964] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:21.087047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:21.089221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:21.095507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:21.096080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:21.096526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.097230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:21.097752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:21.098186 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.129557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d2da8839-8496-4933-995e-9f7224f59200 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046350 Sep 18 06:30:21.130156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 471/965] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:21 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:21.172789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:21.173396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:21.180609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:21.181096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:21.181540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.182244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:21.182733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:21.183165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.300084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:21.300705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:21.306703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:21.307190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:21.307624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.308318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:21.308777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:21.309200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:21.315849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:21.324252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:21.325862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:21.326888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ac7fdc3e-0520-4f71-8eb7-17dbc21560fd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158026 Sep 18 06:30:21.327448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 495/966] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:21 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:21.441964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:21.449760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-50ff4583-cb62-4820-86c9-ae62c087bdd3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154146 Sep 18 06:30:21.450324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 472/967] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:21 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:22.020310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:22.022366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:22.028782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.029252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.029688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.030370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.030846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.031273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.060132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-66c37218-6ab2-4ce1-9c5a-b7ab1b6a8433 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044629 Sep 18 06:30:22.060690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 496/968] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:22 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:22.065709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:22.067893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:22.074298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.074827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.075269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.075962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.076423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.076856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.106275 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7a4802b4-a3e5-4aa2-af69-bea06fcd3336 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044791 Sep 18 06:30:22.106853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 473/969] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:22 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:22.142902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:22.145041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:22.150981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.151465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.151901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.152584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.153035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.153460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.181969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6e1b99f2-ed54-4a14-bdb3-1bea88981c5c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043551 Sep 18 06:30:22.182505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 497/970] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:22 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:22.352829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:22.353376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:22.359422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.359906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.360348 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.361037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.361495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.361920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.471818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:22.472426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:22.478371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.478886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.479327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.480013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:22.480473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:22.480895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:22.488658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:22.497802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:22.499256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:22.500408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a0aaf8fb-c736-4823-ac81-bf9dfcffa650 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151525 Sep 18 06:30:22.500957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 474/971] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:22 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:22.614318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:22.622225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-afb307f6-9b36-4e80-b53d-8578883db28f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154169 Sep 18 06:30:22.622825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 498/972] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:22 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:23.072487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:23.074654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:23.081039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.081516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.081957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.082675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.083130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.083555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.107894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:23.113823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9df459bc-b7a3-4a4c-a854-bcf2427bb85f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045288 Sep 18 06:30:23.114373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 475/973] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:23 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:23.118469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:23.120677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:23.127458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.127944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.128385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.129072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.129531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.129962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.159239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-573dec78-c079-4c08-993e-a37c4fb189bd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044814 Sep 18 06:30:23.159790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 499/974] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:23.194137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:23.196324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:23.202357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.202879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.203315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.204058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.204513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.204937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.233267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-52f84578-c48f-4cdb-b766-341c90e1f343 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043030 Sep 18 06:30:23.233795 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 476/975] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:23 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:23.526909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:23.527517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:23.534034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.534513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.534980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.535664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.536118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.536547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.645737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:23.646454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:23.652405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.652884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.653317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.654003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:23.654452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:23.654935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:23.662937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:23.671480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:23.672919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:23.674082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ecb0b56d-c35e-4f75-85b0-8642e605e736 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151188 Sep 18 06:30:23.674653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 500/976] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:23 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:23.781987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:23.790284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a9e0b278-a3b3-4e4f-b56d-8ce35786d0bc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148908 Sep 18 06:30:23.790889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 477/977] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:23 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:24.126428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:24.128490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:24.134881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.135357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.135782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.136455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.136897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.137309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.165839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9bc3acbf-5095-4008-89b8-741a7b419907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043745 Sep 18 06:30:24.166394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 501/978] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:24 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:24.170962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:24.173088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:24.180408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.180888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.181317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.181994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.182439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.182884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.211530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b4d5287d-3927-40ae-b9fe-0e52649b1e1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044608 Sep 18 06:30:24.212061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 478/979] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:24 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:24.246110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:24.248260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:24.255009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.255485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.255914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.256586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.257034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.257458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.285113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-646cd1f3-1f15-47c4-9a46-eb248703a90f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042929 Sep 18 06:30:24.285653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 502/980] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:24 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:24.699333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:24.699891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:24.705731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.706205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.706688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.707365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.707817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.708232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.813239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:24.813843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:24.819766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.820245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.820677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.821355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:24.821804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:24.822222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:24.838719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:24.847888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:24.849376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:24.850616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-55a65977-8e45-4ba6-8947-6ec47c85a2b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155362 Sep 18 06:30:24.851199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 479/981] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:24 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:24.950384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:24.958483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-36bcea45-2d40-472a-8f71-be2913a2631e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149054 Sep 18 06:30:24.959069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 503/982] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:24 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:25.177305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:25.179483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:25.185747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.186333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.186794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.187476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.187922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.188341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.216967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6e0fb84e-15a0-4399-8f52-6234646802ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043546 Sep 18 06:30:25.217471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 480/983] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:25 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:25.224338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:25.226445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:25.232753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.233229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.233658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.234364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.234841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.235263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.265270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-560c66e5-96e6-4b9b-b22d-9d3a30b5db86 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045289 Sep 18 06:30:25.265820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 504/984] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:25.297307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:25.299534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:25.305319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.305801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.306234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.306942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.307398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.307816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.336807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d234ba2c-a372-408d-92f8-b195d0fc945f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043474 Sep 18 06:30:25.337344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 481/985] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:25 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:25.877733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:25.878330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:25.880227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:25.885693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.886115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.886559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.887255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.887699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.888122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.981812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:25.982427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:25.988736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.989217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.989650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:25.990346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:25.990825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:25.991286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:26.027215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:26.036969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:26.038397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:26.039594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dbacf42d-2780-4d5b-b634-c436e9ecadac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.166655 Sep 18 06:30:26.040154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 505/986] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:25 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:26.122647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:26.130736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-42029468-2c83-413f-a44d-2318596e9b7f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152767 Sep 18 06:30:26.131294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 482/987] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:25 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:26.229394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:26.231566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:26.238736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:26.239234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:26.239662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:26.240367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:26.240817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:26.241242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:26.243667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:26.272565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8bd105ad-18f7-45f3-845c-6659cd699e89 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046173 Sep 18 06:30:26.273791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 506/988] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:26 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:26.279188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:26.281326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:26.287941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:26.288546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:26.288983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:26.289678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:26.290122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:26.290541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:26.322422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b44cf4b6-9fe3-479b-ba61-546dd8f114dd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048632 Sep 18 06:30:26.323012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 483/989] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:26.349922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:26.352195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:26.358142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:26.358644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:26.359079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:26.359747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:26.360189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:26.360728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:26.390139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-383ca488-f9e8-41e6-922a-130901aa19aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044776 Sep 18 06:30:26.390711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 507/990] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:26 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:27.065178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:27.066276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:27.068199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:27.073419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.073911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.074347 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.075090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.075724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.076157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.153925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:27.155790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:27.161510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.162066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.162571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.163365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.163889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.164408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.217895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:27.227883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:27.229555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:27.230785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-36765959-cd51-4056-aacc-2e385ff9087b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.169460 Sep 18 06:30:27.231484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 484/991] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:27 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:27.284169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:27.286434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:27.293049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.293546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.293985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.295042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.295158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.295578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.311889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:27.320235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4b4d1aeb-1515-409c-a668-dceca9e5eb72 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.170845 Sep 18 06:30:27.320793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 508/992] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:27 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:27.328723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0b2f098c-f7ec-4c48-8c1c-64a1c5adf492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048466 Sep 18 06:30:27.329887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 485/993] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:27 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:27.338800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:27.340935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:27.346896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.347375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.347809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.348490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.348936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.349362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.381062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7266f7f0-8d2a-4446-ba8b-d3e10ad61133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046133 Sep 18 06:30:27.381616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 509/994] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:27.402485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:27.404674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:27.410510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.411022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.411456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.412139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:27.412590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:27.413013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:27.442792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4a515b75-e72b-4caa-9843-105784e0604c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044192 Sep 18 06:30:27.443341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 486/995] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:27 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:28.257486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:28.258148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:28.264736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.265218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.265660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.266348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.266830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.267262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.341487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:28.343709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:28.350350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.350874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.351319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.352025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.352485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.352920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.385682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-939406b4-52ba-41b3-acb1-0dbe1fe105c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048100 Sep 18 06:30:28.386451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 487/996] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:28 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:28.390993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:28.391599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:28.396269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:28.397972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.398459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.398926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.399615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.400066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.400486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.405250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:28.406846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:28.408084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a3562fdf-b6a8-4265-9723-2bd0ea97e29b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155111 Sep 18 06:30:28.408716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 510/997] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:28 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:28.413414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:28.415624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:28.421540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.422144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.422700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.423446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.423961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.424451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.456566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a4eb2da1-be23-4096-bd11-fc6c1955738f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045986 Sep 18 06:30:28.461410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 511/998] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:28 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:28.461916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:28.464082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:28.470195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.470713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.471147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.471838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:28.472289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:28.472709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:28.503470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c594355b-0a6f-4beb-aca2-44fa0b47bc7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044714 Sep 18 06:30:28.504219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 512/999] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:28 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:28.538185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:28.547488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a4e65f9f-2610-40f9-a8b7-ce7acb4859ad tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.160175 Sep 18 06:30:28.548055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 488/1000] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:28 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:29.400458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:29.402590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:29.409663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.410175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.410637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.411314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.411761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.412180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.432759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:29.433406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:29.441115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cd13c31b-fdeb-4ff7-b8b1-9efdf1103051 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044579 Sep 18 06:30:29.441857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 513/1001] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:29 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:29.444175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.444664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.445099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.445799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.446258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.446707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.468994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:29.471172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:29.477960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.478448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.478922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.479617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.480061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.480479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.510919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f29f6f57-d20f-4055-be1b-dc224e8b5663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045762 Sep 18 06:30:29.511493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 514/1002] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:29.516059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:29.518693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:29.524444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.524923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.525363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.526052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.526503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.526953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.556941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b1bb5412-2577-4f0c-8a92-debc968e4d4b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044390 Sep 18 06:30:29.557706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 515/1003] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:29 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:29.570313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:29.570955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:29.576763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.577247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.577681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.578368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:29.578847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:29.579273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:29.585068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:29.593475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:29.594956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:29.596102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e6d1e178-85e0-416c-94d2-35712e5a6a53 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.167268 Sep 18 06:30:29.597487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 489/1004] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:29 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:29.713012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:29.720913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9d682746-5e24-489a-b34b-ce304b58e6d0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154317 Sep 18 06:30:29.722060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 516/1005] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:29 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:30.453755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:30.456015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:30.462519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.463039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.463481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.464180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.464632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.465061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.494177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-32444b3e-9196-47a8-babe-437785aaa7df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044480 Sep 18 06:30:30.494739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 490/1006] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:30 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:30.523587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:30.525837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:30.532474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.532962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.533402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.534613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.535080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.535505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.565830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-71e5e900-5d48-41a7-8ce1-04309d916494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046461 Sep 18 06:30:30.566415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 517/1007] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:30 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:30.569258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:30.571423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:30.577469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.577952 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.578385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.579114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.579567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.580043 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.608555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3dd124eb-0553-44f0-ada4-dd4c0df39e86 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043223 Sep 18 06:30:30.609097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 491/1008] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:30 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:30.622534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:30.623253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:30.629457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.629939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.630373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.631087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.631542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.631968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.744672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:30.745279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:30.751440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.751925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.752358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.753041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:30.753487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:30.753907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:30.762012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:30.770830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:30.771492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:30.772442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:30.773679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a0e47703-fd57-4194-8271-f17d991dc518 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155087 Sep 18 06:30:30.774237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 518/1009] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:30 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:30.884721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:30.892434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f3359da8-5259-4b69-86f3-5688cbdc3266 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151772 Sep 18 06:30:30.892989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 492/1010] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:30 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:31.507430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:31.509592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:31.515726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.516206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.516639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.517320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.517772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.518239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.547171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c170b8af-4442-4d97-9dad-9439485d7087 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044336 Sep 18 06:30:31.550715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 519/1011] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:31 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:31.580938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:31.583157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:31.589574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.590054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.590485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.591209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.591660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.592084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.621372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:31.624195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:31.630601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.631158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.631600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.632885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.633493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.633938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.637574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-96b94408-00d6-4ac3-88aa-c6b3e3baca3c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061216 Sep 18 06:30:31.638131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 493/1012] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:31.668387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-90ee1d72-1b9b-4592-a34e-b7399458aa3c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050892 Sep 18 06:30:31.668943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 520/1013] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:31 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:31.799309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:31.799915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:31.805834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.806312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.806772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.807453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.807903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.808318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.915906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:31.916514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:31.927778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.928278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.928730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.929485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:31.929941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:31.930437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:31.942880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:31.951852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:31.953305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:31.954533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-99745d35-d46a-4ec1-aecb-d3558cf986b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.159233 Sep 18 06:30:31.955089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 494/1014] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:31 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:32.070914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:32.079065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-be4819df-f23f-405c-a5d8-ffbfb54f3970 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.167008 Sep 18 06:30:32.079634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 521/1015] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:31 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:32.560268 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:32.562432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:32.569090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:32.569576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:32.570015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:32.570750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:32.571199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:32.571627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:32.600641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-828dbbff-1fc8-45d1-8738-8dff3a9ff961 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044310 Sep 18 06:30:32.601245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 495/1016] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:32 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:32.649472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:32.651702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:32.658154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:32.658693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:32.659132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:32.659821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:32.660281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:32.660706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:32.681780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:32.684182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:32.689318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:32.691987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:32.694878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:32.694878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:32.694878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:32.694878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:32.695202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:32.697629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8d6e2dbd-ed7b-472b-9e97-1fa339935b45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052154 Sep 18 06:30:32.698860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 522/1017] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:32 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:32.724785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8a252a66-9916-4db2-b9ad-4e6c016451dc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047233 Sep 18 06:30:32.725333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 496/1018] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:32 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:32.981281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:32.981918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:32.990199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:32.990709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:32.991140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:32.991822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:32.992267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:32.992685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.103306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:33.104028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:33.109902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:33.110382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:33.110854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.111541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:33.112633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:33.113056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.122828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:33.131719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:33.133160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:33.134323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-261be2e8-fe5c-4b1b-835f-fce0af0180f6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157299 Sep 18 06:30:33.135106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 523/1019] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:32 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:33.239633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:33.247345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-280629f7-0021-41d5-8ec4-e92b0926b9b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147904 Sep 18 06:30:33.247900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 497/1020] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:33 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:33.613131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:33.615242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:33.621163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:33.621641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:33.622076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.622790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:33.623234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:33.623655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.653547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f5097b84-a266-48ec-b3d5-37c6bb2a2908 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044426 Sep 18 06:30:33.654147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 524/1021] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:33 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:33.709754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:33.711958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:33.718330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:33.718849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:33.719280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.719959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:33.720409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:33.720829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.737157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:33.739365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:33.745910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:33.746471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:33.747025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.747829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:33.748282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:33.748754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:33.750250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-74566cc5-053f-413c-86d8-932cf485897c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044900 Sep 18 06:30:33.751093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 498/1022] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:33 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:33.778774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0ac21a74-1b8a-4421-9df3-c8426b93dfd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045466 Sep 18 06:30:33.779471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 525/1023] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:33 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:34.159054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:34.159662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:34.165524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.166003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.166430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.167139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.167587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.168006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.270526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:34.271172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:34.277142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.277623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.278057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.278766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.279216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.279636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.295375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:34.304768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:34.306248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:34.307472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cc180a2b-c1b8-43bb-8493-96438cd35951 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152330 Sep 18 06:30:34.308028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 499/1024] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:34 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:34.416161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:34.423861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-da84ec13-7538-4df6-b70d-e7738a902975 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.157411 Sep 18 06:30:34.425060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 526/1025] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:34 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:34.666024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:34.668199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:34.674150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.674669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.675108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.675795 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.676247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.676672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.704949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b3bfca4e-b513-4596-89fa-296751a90689 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042977 Sep 18 06:30:34.706028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 500/1026] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:34 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:34.763395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:34.765525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:34.772457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.772930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.773363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.774044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.774493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.774945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.790980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:34.793328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:34.799783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.800346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.800798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.801501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:34.802087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:34.802519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:34.805343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f7bb90c7-71ac-4076-ba05-d9f7f3114df8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045898 Sep 18 06:30:34.806132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 527/1027] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:34.831922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-98d29931-e9aa-416b-8bac-24b1c823c33d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044852 Sep 18 06:30:34.832481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 501/1028] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:34 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:35.332400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:35.333004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:35.339079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.339559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.339995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.340717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.341166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.341589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.447470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:35.448077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:35.454250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.454780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.455215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.456576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.457028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.457452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.470780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:35.479947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:35.481390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:35.482556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-42d97ad2-4986-4b48-b291-fe67ad2b814e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154063 Sep 18 06:30:35.483146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 528/1029] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:35 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:35.589782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:35.598039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9f443097-0cff-421d-a753-0eccd315fb05 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154412 Sep 18 06:30:35.598619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 502/1030] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:35 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:35.718102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:35.720406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:35.726933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.727411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.727844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.728518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.728966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.729385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.758652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a17b39bb-0657-43d1-be35-faa4beb9937a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044525 Sep 18 06:30:35.759197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 529/1031] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:35 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:35.818155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:35.820483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:35.827004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.827486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.827915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.828596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.829046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.829464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.844570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:35.846690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:35.853115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.853607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.854047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.854893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:35.855232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:35.855657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:35.860616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-17abac91-2fe7-4bf9-a04e-bb8fa9405053 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046806 Sep 18 06:30:35.861178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 503/1032] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:35.885388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1ccb59aa-c358-4f4d-94fd-4749354081e3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044890 Sep 18 06:30:35.886001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 530/1033] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:35 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:36.508300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:36.508903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:36.514886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.515363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.515793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.516470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.516913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.517332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.621180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:36.621878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:36.628562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.629044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.629471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.630150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.630645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.631072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.646083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:36.659485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:36.660918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:36.662085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c59aab42-93f8-4406-96c1-f575d1bf301d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157702 Sep 18 06:30:36.662658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 504/1034] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:36 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:36.764650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:36.771713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:36.773466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6557d5e4-b270-48cb-8f4e-6160ce8ebbb0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.156131 Sep 18 06:30:36.773845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:36.774035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 531/1035] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:36 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:36.779719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.780192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.780621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.781306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.781753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.782174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.811661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-de2666ce-0b3b-469f-a076-78f76fd2b033 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044518 Sep 18 06:30:36.812207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 505/1036] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:36 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:36.872555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:36.874736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:36.881865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.882371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.882842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.883524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.883976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.884402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.898695 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:36.900860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:36.907203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.907681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.908145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.908834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:36.909279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:36.909696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:36.914844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4dc2bd42-912c-4010-9001-1fb0c8747204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046153 Sep 18 06:30:36.915409 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 532/1037] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:36.938536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e6d1041a-efa9-4a76-86a7-7bffc8ac0eda tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044127 Sep 18 06:30:36.939097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 506/1038] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:36 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:37.687046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:37.687648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:37.693693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.694167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.694622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.695302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.695751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.696167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.796593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:37.797336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:37.803276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.803767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.804207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.804901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.805359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.805787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.822927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:37.831898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:37.833329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:37.834479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-91302e5c-b173-4b3f-a0c3-ef6e8428e0ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151409 Sep 18 06:30:37.835069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 533/1039] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:37 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:37.839513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:37.841640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:37.850017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.850017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.850017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.850718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.851317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.851737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.882711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2174bd99-f0e2-483b-b1a9-65deffb0cb8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046489 Sep 18 06:30:37.883386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 534/1040] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:37 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:37.927425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:37.929657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:37.935732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.936218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.936660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.937401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.937804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.938233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.940886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:37.948903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9b26e9d5-918f-40f9-a96d-1e55256f263c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.156152 Sep 18 06:30:37.949464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 507/1041] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:37 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:37.954735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:37.958182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:37.964535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.965032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.965472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.966179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:37.966689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:37.967125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:37.972396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a7248413-b810-43d1-b1cd-9ce07a17f7bf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048886 Sep 18 06:30:37.972977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 535/1042] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:37 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:37.997418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d4dde84d-101c-4f2a-9ccb-39dc70af20b6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046946 Sep 18 06:30:37.997965 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 508/1043] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:37 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:38.861449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:38.862491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:38.868545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:38.869029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:38.869466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:38.870169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:38.870647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:38.871076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:38.895092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:38.897227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:38.903447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:38.903932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:38.904426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:38.905054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:38.905511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:38.905932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:38.911563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:38.938840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ebe31ec2-f705-4489-b503-36beedf4e7bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047580 Sep 18 06:30:38.939404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 509/1044] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:38 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:38.973085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:38.973722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:38.979851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:38.980453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:38.980893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:38.981582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:38.982036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:38.982456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:39.003584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:39.012321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:39.013749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:39.014930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-97a4ef16-6cd7-4006-b4cc-764905b1be01 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157562 Sep 18 06:30:39.015792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 536/1045] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:38 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:39.020993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:39.023223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:39.031928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:39.032413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:39.032948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:39.034146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:39.034600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:39.035038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:39.064668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-90d0005a-a035-4ca5-903e-b4e8d471a82f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047827 Sep 18 06:30:39.065232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 537/1046] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:39 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:39.069917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:39.072223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:39.077879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:39.078359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:39.078905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:39.079568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:39.080022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:39.080480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:39.109023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-53895c7a-c01b-4d0a-854f-b7cfbdd73486 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042714 Sep 18 06:30:39.109579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 538/1047] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:39 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:39.115627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:39.123258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b958f4ad-84f6-4f04-a26e-06145c745fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154538 Sep 18 06:30:39.123809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 510/1048] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:38 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:39.951357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:39.953519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:39.959907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:39.960382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:39.960815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:39.961487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:39.961931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:39.962349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:39.975809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:39.992241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-12421776-13d6-4d8e-b17c-0d2d630c9031 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044845 Sep 18 06:30:39.992790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 539/1049] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:39 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:40.040142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:40.040749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:40.047310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:40.047789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:40.048221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:40.048899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:40.049348 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:40.050300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:40.076931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:40.079168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:40.085189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:40.085671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:40.086128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:40.086830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:40.087284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:40.087709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:40.117758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-67f572bb-8682-46e8-a0d4-9021a529ec2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044737 Sep 18 06:30:40.118346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 540/1050] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:40.122951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:40.125054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:40.131780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:40.132256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:40.132687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:40.133369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:40.133816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:40.134234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:40.163017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5ee9a5ce-1268-4485-bb53-e9b232aff322 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043580 Sep 18 06:30:40.163584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 541/1051] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:40 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:40.167897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:40.168492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:40.173986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:40.174460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:40.175406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:40.176083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:40.176530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:40.176950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:40.181008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:40.191543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:40.192976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:40.194138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dd2464ca-aeaf-4814-8602-9e8b9b406ff8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157856 Sep 18 06:30:40.194714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 511/1052] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:40 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:40.310319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:40.317983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-39894031-8c97-4bfc-b86d-6e95b5330829 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153473 Sep 18 06:30:40.318607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 542/1053] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:40 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:41.005336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:41.007535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:41.013484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.013964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.014397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.015103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.015553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.015974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.047718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-40678e72-4543-4f64-bdbc-00f6015240f5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047116 Sep 18 06:30:41.048375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 512/1054] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:40 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:41.130502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:41.132671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:41.139193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.139675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.140107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.140804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.141255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.141680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.171814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-74a9df80-e71d-413b-9f7c-9846aab9b04f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045252 Sep 18 06:30:41.172372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 543/1055] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:41.176084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:41.178222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:41.184166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.184645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.185078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.185751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.186203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.186654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.215328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cbc2e9b1-b049-4d0a-8606-d572167d7400 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043891 Sep 18 06:30:41.215869 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 513/1056] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:41 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:41.219022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:41.219632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:41.225993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.226469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.226942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.227622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.228070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.228490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.341173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:41.341804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:41.348542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.348977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.349693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.351001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:41.351659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:41.351934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:41.356710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:41.368241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:41.369741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:41.371009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-76a61991-f58d-42ca-bbe1-f86b71ab0745 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155711 Sep 18 06:30:41.371602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 544/1057] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:41 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:41.488512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:41.496393 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fb8fbb8e-1371-4dbb-9865-d887ae5ee31c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.159190 Sep 18 06:30:41.496953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 514/1058] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:41 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:42.062448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:42.064729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:42.071223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.071704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.072138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.072821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.073273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.073696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.102932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2121609c-d0dc-47a8-b22f-575ae7b0d89b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044738 Sep 18 06:30:42.103485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 545/1059] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:42 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:42.184843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:42.187057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:42.194041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.194525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.194998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.195683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.196140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.196571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.225416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8b548008-6f51-4737-80f6-295aa472e391 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044534 Sep 18 06:30:42.225966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 515/1060] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:42.228008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:42.230158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:42.236429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.236906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.237831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.238508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.239107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.239706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.270090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1c0580f5-62cb-49bf-8c50-a7ed66ab73f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045967 Sep 18 06:30:42.270656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 546/1061] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:42 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:42.396978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:42.397619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:42.403573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.404056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.404499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.405184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.405640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.406063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.520874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:42.523371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:42.529548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.530025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.530455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.531196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:42.531647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:42.532065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:42.538375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:42.546968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:42.548404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:42.549573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-172154fa-3d75-4ca8-8579-f88480a2ee21 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157040 Sep 18 06:30:42.550128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 516/1062] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:42 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:42.663637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:42.671263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0061dd2f-17d8-401c-8b84-d133fa1c6553 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155581 Sep 18 06:30:42.671819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 547/1063] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:42 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:43.116119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:43.118211 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:43.124176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.124662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.125100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.126051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.126498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.126945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.155902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cc30a2c1-abf2-4ab9-969e-401f97495e9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044399 Sep 18 06:30:43.156436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 517/1064] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:43 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:43.238215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:43.240375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:43.247015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.247493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.247929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.248612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.249062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.249484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.279718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6070cd0a-102e-4762-af3f-ddd91534b3af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045930 Sep 18 06:30:43.280271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 548/1065] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:43.284392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:43.286539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:43.292961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.293443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.293877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.294566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.295039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.295460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.324044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fdca9bea-6755-424e-b4fe-b5f36bc2cef4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045576 Sep 18 06:30:43.324585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 518/1066] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:43 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:43.574991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:43.575598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:43.581594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.582069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.582503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.583208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.583659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.584084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.693737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:43.694428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:43.700274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.700758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.701191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.701876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:43.702327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:43.702771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:43.712040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:43.720743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:43.722715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:43.723881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7d8851c9-c68e-4fe0-adea-c37000072b43 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152792 Sep 18 06:30:43.724444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 549/1067] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:43 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:43.835246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:43.843073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c47a3fd2-a161-4a60-a05a-65f616c39074 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153272 Sep 18 06:30:43.843628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 519/1068] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:43 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:44.168296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:44.170416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:44.176345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.176818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.177249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.177921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.178366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.178815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.207929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-93907388-7539-4a02-af69-12aa0bb8b2d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043629 Sep 18 06:30:44.208466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 550/1069] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:44 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:44.292636 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:44.294812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:44.301446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.302016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.302459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.303164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.303624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.304654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.336729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-17f1e6ff-e7d8-4ada-b64a-b4ccee324f02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047818 Sep 18 06:30:44.338220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 520/1070] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:44.437194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:44.439506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:44.446141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.446656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.447098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.447790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.448243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.448672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.478549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-afd3fd32-eb81-4b7f-9a20-f7836fee28c2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.145448 Sep 18 06:30:44.479123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 551/1071] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:44 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:44.749422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:44.749957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:44.756785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.757265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.757699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.758378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.758852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.759273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.867675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:44.868289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:44.874188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.874698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.875134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.876077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:44.876528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:44.876948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:44.886194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:44.894951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:44.896412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:44.897564 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6ff002f9-6106-41a3-8484-78063f1d897b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152178 Sep 18 06:30:44.898112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 521/1072] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:44 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:30:45.005211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:45.012936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e090c29b-a6a7-47c2-96cc-607fcefa1d99 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149302 Sep 18 06:30:45.013496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 552/1073] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:44 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:45.220228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:45.223117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:45.228984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:45.229461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:45.229894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:45.230596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:45.231044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:45.231465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:45.260966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-086ed0ec-5f9b-4181-90f7-da32d50c1af4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044586 Sep 18 06:30:45.261759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 522/1074] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:45 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:45.349688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:45.351963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:45.358429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:45.358939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:45.359376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:45.360060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:45.360513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:45.360937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:45.390308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7bae643d-495d-4e2c-8a65-afb6b4034cf5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045239 Sep 18 06:30:45.390896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 553/1075] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:45 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:45.491330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:45.493532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:45.500141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:45.500880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:45.501313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:45.502007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:45.502987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:45.503413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:45.535684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3b22f477-a366-4233-914d-9e217a5d0fb7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048256 Sep 18 06:30:45.536166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 523/1076] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:45 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:45.925902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:45.926523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:45.932549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:45.933030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:45.933468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:45.934156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:45.934622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:45.935051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.036278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:46.036930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:46.042883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:46.043367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:46.043809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.044499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:46.044956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:46.045383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.067877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:46.076493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:46.077938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:46.079139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-afde036f-de1a-4e24-aeec-98d69631031c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.159315 Sep 18 06:30:46.079701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 554/1077] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:45 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:46.175661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:46.183381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9069bde2-5c47-4249-8f66-1de04e2d01e8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151068 Sep 18 06:30:46.183888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 524/1078] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:46 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:46.273031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:46.275203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:46.277023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:46.282138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:46.282645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:46.283082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.284288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:46.284740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:46.285167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.315291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-afa5d623-bd96-4a97-a21f-2cf00bf4af37 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046178 Sep 18 06:30:46.315841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 555/1079] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:46 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:46.403210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:46.405419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:46.411840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:46.412327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:46.412762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.413448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:46.413903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:46.414323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.440575 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:46.445935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5a9c07f1-ef3d-4a86-9c40-cc54cfce33b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046641 Sep 18 06:30:46.446513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 525/1080] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:46.547801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:46.550563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:46.557066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:46.557541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:46.557970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.558673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:46.559155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:46.559578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:46.563150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:46.588476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7772328a-a39e-46d9-ae17-af19b4b0534e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044815 Sep 18 06:30:46.589029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 556/1081] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:46 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:47.104625 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:47.105159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:47.111787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.112306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.112751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.113453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.113914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.114337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.207296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:47.207903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:47.213860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.214340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.214816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.215543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.215997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.216420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.248631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:47.258240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:47.259705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:47.260875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ccb76a84-3538-41a5-8541-36317df56a22 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.161271 Sep 18 06:30:47.261432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 526/1082] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:47 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:47.328104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:47.331058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:47.337116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.337604 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.338047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.338777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.339234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.339659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.345911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:47.353630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8f15c4a7-1af9-468d-add5-36076a84ba27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150706 Sep 18 06:30:47.354213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 557/1083] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:47 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:47.369806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c90e1c2b-3639-48ab-a59b-3b5198ff2f77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045703 Sep 18 06:30:47.370399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 527/1084] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:47 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:47.459005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:47.460979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:47.469215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.469710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.470146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.470879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.471328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.471752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.503668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fabfd50a-0e1e-4ad4-91f4-207cdb6f066d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048981 Sep 18 06:30:47.504238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 558/1085] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:47.601203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:47.603452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:47.609531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.609939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.610386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.611138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:47.612143 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:47.612516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:47.644530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-20ecd7d5-e452-41a7-ae13-5865bbc9559e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047175 Sep 18 06:30:47.649238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 528/1086] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:47 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:48.287133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:48.287871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:48.294459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.294976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.295415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.296110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.296567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.296994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.378413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:48.379111 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:48.386082 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.386967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.387407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.388136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.388587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.389040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.443566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:48.457194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:48.458671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:48.460134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f1d45bc3-8687-480c-b6fe-46d12acf8e4c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.177012 Sep 18 06:30:48.463758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 559/1087] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:48 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:48.474124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:48.476428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:48.483122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.483628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.484067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.485001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.485460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.485889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.519749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e62631bf-b699-401b-90f3-7fd87a0161b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050552 Sep 18 06:30:48.520325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 560/1088] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:48 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:48.527513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:48.530069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:48.536681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.537284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.537669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.538365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.538974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:48.539118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.539404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.547420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ddf74829-636f-474e-b268-6669db01f0a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.173784 Sep 18 06:30:48.548269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 529/1089] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:48 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:48.573964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-67189d0f-9e95-47f4-9a13-a24dbfcf52e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050226 Sep 18 06:30:48.574524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 561/1090] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:48.657160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:48.659948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:48.666207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.666838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.667279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.667978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:48.668428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:48.668848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:48.699104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-62449e25-5b97-409a-a22f-65dafa89ea4a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045874 Sep 18 06:30:48.699664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 530/1091] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:48 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:49.488209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:49.488818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:49.494952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.495435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.495869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.496563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.497014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.497439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.537825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:49.540173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:49.546936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.547430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.547867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.548573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.549074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.549500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.591167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2a31abb2-a9fe-4de1-a485-28094fead078 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062825 Sep 18 06:30:49.592058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 531/1092] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:49 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:49.596518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:49.597159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:49.603193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.604003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.604443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.605141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.605598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.606171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.638048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:49.647890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:49.650760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:49.652133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d131f13a-936d-49c4-9854-503eb9d99e1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.168102 Sep 18 06:30:49.653658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 562/1093] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:49 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:49.658435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:49.660581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:49.667142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.667624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.668055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.668743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.669194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.670544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.712907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a31c82c3-d255-45f4-93ac-63935eb91f2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058143 Sep 18 06:30:49.713698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 563/1094] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:49.718426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:49.720581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:49.726431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.726950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.727692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.728402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:49.728857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:49.729280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:49.751075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:49.760983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-58ad05b8-9c8d-4024-b0df-c3565f525f01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046443 Sep 18 06:30:49.761561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 564/1095] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:49 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:49.761843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b2700289-8a51-4e8d-8379-eca01558e816 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.168267 Sep 18 06:30:49.762555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 532/1096] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:49 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:50.603890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:50.606045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:50.612813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.613436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.613871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.614566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.615038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.615521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.647264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-37e79707-e60c-4660-b7e5-1383b7db13e4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047136 Sep 18 06:30:50.647649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 565/1097] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:50 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:50.679047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:50.679864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:50.687631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.688123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.688565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.689680 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.690144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.690674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.726955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:50.729370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:50.735687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.736167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.736684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.737298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.737749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.738302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.772788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d0bf11fe-d3b8-4e61-a509-8cbaecfc2676 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049677 Sep 18 06:30:50.773566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 566/1098] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:50.782144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:50.784909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:50.791829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.792697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.793357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.794427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.795105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.795921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.827093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c5efc77c-5cc8-4d9e-a72a-4efaa8f5db11 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052670 Sep 18 06:30:50.827948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 567/1099] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:50 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:50.829265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:50.834841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:50.835459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:50.839213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:50.841257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:50.841754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.842299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.842459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f9f8da03-2917-48a8-b368-889d6f6352e2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.169041 Sep 18 06:30:50.842798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.843318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 533/1100] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:50 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:50.843510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:50.843938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:50.844354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:50.979438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:50.987432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e16b7fb5-b829-47b9-a994-e32fe1dda66d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.158329 Sep 18 06:30:50.988027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 568/1101] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:50 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:51.660104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:51.662372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:51.664395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:51.670147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:51.670734 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:51.671179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:51.671879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:51.672485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:51.672917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:51.704516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e222d4a6-176a-4c2a-a383-cf2c7aacaafb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048838 Sep 18 06:30:51.705171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 534/1102] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:51 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:51.785904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:51.788069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:51.794861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:51.795352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:51.795787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:51.796478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:51.796932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:51.797385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:51.828460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b589ab65-9451-4e1e-bebd-4fba7407cc28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047123 Sep 18 06:30:51.829022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 569/1103] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:51.839954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:51.842107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:51.848895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:51.849387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:51.849829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:51.850527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:51.851029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:51.851459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:51.868570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:51.869740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:51.875498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:51.875982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:51.876417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:51.877156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:51.877929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:51.878349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:51.883435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-628c2fb2-60e2-43f5-90a2-8464b2f3f6df tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047522 Sep 18 06:30:51.884011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 535/1104] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:51 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:52.011926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:52.012535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:52.013999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:52.019072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:52.019559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:52.019999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:52.020683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:52.021262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:52.021821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:52.023803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:52.025058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:52.026252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-63c8b723-8156-4ad1-9e45-bd3609e37278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.161876 Sep 18 06:30:52.027150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 570/1105] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:51 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:52.170682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:52.178826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8065f1c1-cbcf-4abc-920e-08b65c77b5a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.170797 Sep 18 06:30:52.179394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 536/1106] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:52 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:52.719315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:52.721504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:52.728361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:52.728851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:52.729318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:52.729983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:52.730445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:52.730964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:52.762161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-76e42257-006b-4ef3-aa76-25738a2ce58d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046936 Sep 18 06:30:52.762819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 571/1107] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:52 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:52.841039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:52.843226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:52.849775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:52.850261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:52.850745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:52.851424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:52.851884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:52.852309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:52.884067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8f309fc5-bd8f-40a7-9e72-aa9da263a86f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046997 Sep 18 06:30:52.887928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 537/1108] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:52 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:52.895081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:52.897220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:52.903724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:52.904212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:52.904647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:52.905334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:52.905782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:52.906200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:52.932867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:52.938523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-20dc0de6-ee2d-4729-a507-ebb73710998c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047507 Sep 18 06:30:52.939153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 572/1109] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:52 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:53.056983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:53.057614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:53.064031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.064517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.064957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.065654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.066107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.066532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.205497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:53.205497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:53.205722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:53.209070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.209849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.210350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.211139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.211586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.212004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.213416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:53.214924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:53.216101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-61b744ea-8e15-40e0-903b-a54f44a30fa2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.163420 Sep 18 06:30:53.217670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 538/1110] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:53 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:53.350777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:53.358709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0c553f05-0ed0-4259-8e14-fa83fb3aa02b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.160845 Sep 18 06:30:53.359301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 573/1111] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:53 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:53.774996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:53.777240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:53.783450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.783940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.784400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.785089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.785548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.785972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.818233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-42334cad-a832-4a86-ac55-93916ca01b82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047226 Sep 18 06:30:53.819159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 539/1112] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:53 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:53.897922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:53.900120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:53.906713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.907188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.907615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.908322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.908763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.909183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.937605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f38b6001-b91e-4216-852c-1cc92804648e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043649 Sep 18 06:30:53.938156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 574/1113] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:53.951537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:53.954157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:53.961783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.962271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.962733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.963430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:53.963885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:53.964314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:53.994144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5c505e7b-ada6-4247-ad47-fb1d7b8a1599 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046444 Sep 18 06:30:53.994912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 540/1114] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:53 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:54.243777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:54.244461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:54.250805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:54.251285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:54.251719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:54.253009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:54.253459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:54.253885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:54.384024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:54.384710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:54.386255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:54.391620 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:54.392227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:54.393128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:54.393883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:54.394377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:54.394892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:54.395602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:54.397048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:54.398213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eee42b2b-0034-4663-926a-5fc2a0985606 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158744 Sep 18 06:30:54.400231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 575/1115] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:54 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:54.527998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:54.536202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ff4c8df2-cf4e-4dc6-a87c-1b5315ee5244 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.157012 Sep 18 06:30:54.536759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 541/1116] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:54 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:54.830690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:54.832755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:54.838704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:54.839187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:54.839621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:54.840302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:54.840754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:54.841176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:54.880188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bf0c3363-5c76-4ac2-b777-051f9898d87f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053588 Sep 18 06:30:54.880750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 576/1117] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:54 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:54.950337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:54.952540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:54.959425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:54.959908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:54.960335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:54.961019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:54.961467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:54.961890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:54.978700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:54.992605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-06a0da6a-ca2f-43da-83e8-2b9097db26e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046183 Sep 18 06:30:54.993151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 542/1118] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:55.006983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:55.009140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:55.015559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:55.016039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:55.016473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:55.017151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:55.018107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:55.018530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:55.047663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aa4f41a8-6b44-4aa1-a0b8-9b471d28529f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044686 Sep 18 06:30:55.048211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 577/1119] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:55 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:55.423293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:55.424064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:55.429979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:55.430454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:55.430916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:55.431597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:55.432044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:55.432464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:55.559254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:55.560313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:55.560922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:55.567417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:55.567896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:55.568323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:55.569008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:55.569118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:55.569451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:55.569868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:55.570524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:55.571748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-84ae9560-febc-46a9-9c99-84b0af81a8f9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152321 Sep 18 06:30:55.572300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 543/1120] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:55 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:55.700985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:55.708608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ea3b24a4-85e0-4030-b8ba-c8adb9bafb29 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152374 Sep 18 06:30:55.709174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 578/1121] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:55 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:55.892741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:55.894921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:55.900794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:55.901274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:55.901703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:55.902381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:55.902850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:55.903270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:55.932032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-46522aef-2b9c-4cf1-bdd1-5871451d3a77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043334 Sep 18 06:30:55.932567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 544/1122] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:55 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:56.006223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:56.008317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:56.014901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.015384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.015821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.016515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.016966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.017389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.048207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fef2579b-a743-4e62-8478-7dbf3ea623c7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046974 Sep 18 06:30:56.049322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 579/1123] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:56.060284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:56.063028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:56.069708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.070271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.070732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.071419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.071864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.072282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.101372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dc362aa0-4828-4e18-b267-802f58d5b8d4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045034 Sep 18 06:30:56.101918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 545/1124] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:56 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:56.596437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:56.597033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:56.602988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.603467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.603900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.604578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.605025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.605442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.730248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:56.731294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:56.731908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:56.738102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.738602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.739022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.739578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:56.739694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.740125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.740540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.741016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:56.742204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2e468168-a065-4ca7-b7a2-cdbbce2629d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149703 Sep 18 06:30:56.742961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 580/1125] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:56 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:56.876286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:56.884013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bd7a14b5-3c3c-4f83-a851-11fb1ef651c5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.156501 Sep 18 06:30:56.884574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 546/1126] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:56 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:56.944212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:56.946325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:56.952279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.952755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.953188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.953871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:56.954314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:56.954843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:56.983579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5dbea806-9ecb-4d5e-b994-2b8e50a748c6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043282 Sep 18 06:30:56.984058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 581/1127] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:56 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:57.061238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:57.063444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:57.069519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:57.069999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:57.070432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:57.071150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:57.071602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:57.072036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:57.100668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6c021759-f5d8-40ad-b3cc-2c738d6d0a5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043353 Sep 18 06:30:57.101200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 547/1128] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:57.113811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:57.116137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:57.122823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:57.123299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:57.123724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:57.124432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:57.125510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:57.126214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:57.156905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-87980d9d-73eb-40cb-be4a-53a115ba2553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047632 Sep 18 06:30:57.157454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 582/1129] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:57 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:57.768773 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:57.769324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:57.775466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:57.775956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:57.776392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:57.777087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:57.777537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:57.777962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:57.903352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:57.907247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:57.908471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:57.912533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:57.913971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:57.914263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:57.914785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:57.915165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9c19d21c-9d86-4f60-b990-cd8fb4fc9d5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150443 Sep 18 06:30:57.915276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:57.915714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 548/1130] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:57 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:57.915905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:57.916359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:57.916910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:57.996769 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:57.998965 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:58.005169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:58.005653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:58.006086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:58.006803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:58.007252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:58.007671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:58.038300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-91520e51-eac5-42d8-8bd9-05e3304444d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045486 Sep 18 06:30:58.038803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 549/1131] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:57 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:58.047776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:58.055483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cff118cd-503d-4ede-af67-b8250cfadb80 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152168 Sep 18 06:30:58.056036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 583/1132] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:57 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:30:58.112950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:58.115105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:58.121414 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:58.121894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:58.122323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:58.123036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:58.123483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:58.123907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:58.152941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b1992ae0-6301-4c75-b564-bd8cc62c1ce1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043992 Sep 18 06:30:58.153476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 550/1133] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:58 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:58.169153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:58.171368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:58.177641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:58.178125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:58.178537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:58.179243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:58.179688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:58.180106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:58.208383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f3d2055a-a676-4549-b515-481403267550 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043146 Sep 18 06:30:58.208913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 584/1134] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:58 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:59.035881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:59.036498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:59.042325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.042849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.043286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.043996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.044448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.044868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.050547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:59.052958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:59.059027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.059509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.059944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.060632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.061077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.061496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.091880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc03c00d-e1ea-46c2-a5f8-edd752af0792 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045113 Sep 18 06:30:59.092689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 585/1135] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:59 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:30:59.096873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:59.097467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:59.102978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.103454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.103882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.104555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.105001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.105420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.210856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:59.219767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:59.221425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:59.222616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-58c1031c-57f7-47e5-92dc-5df7333e9d35 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.286063 Sep 18 06:30:59.223165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 551/1136] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:30:58 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:30:59.227691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:59.229944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:59.235736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.236220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.236657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.237444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.237901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.238321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.256387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:30:59.264082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1cafe75c-41f9-4c1f-b042-6b6400478da5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.170618 Sep 18 06:30:59.264653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 586/1137] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:30:59 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:30:59.269971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7e1d28eb-779d-4b3a-95eb-d1a258e4807f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045747 Sep 18 06:30:59.270139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:30:59.270798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 552/1138] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:59 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:30:59.272893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:30:59.278742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.279218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.279646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.280332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:30:59.280792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:30:59.281208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:30:59.310821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a16c6b63-7628-4ba4-bc71-830d639032bf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045107 Sep 18 06:30:59.311379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 587/1139] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:30:59 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:00.104394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:00.106607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:00.113139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.113614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.114045 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.114751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.115199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.115618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.144669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f87e9a27-0375-42bc-964c-c7e63cb4c3a1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044265 Sep 18 06:31:00.145204 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 553/1140] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:00 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:00.247778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:00.248411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:00.255115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.255598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.256051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.256713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.257264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.257603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.282491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:00.285327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:00.291256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.291774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.292210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.292899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.293351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.293773 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.323851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f77d7236-5135-4f86-99bd-1c4f4c1a081f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045332 Sep 18 06:31:00.324411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 554/1141] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:31:00.328797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:00.330293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:00.335928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.336409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.336837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.337954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.337954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.338368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.416347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:00.424902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:00.426324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:00.427506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6add17dd-4688-4ff4-91c4-8376f7a28487 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.183609 Sep 18 06:31:00.428258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 588/1142] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:00 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:00.432721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:00.434838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:00.440717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.441198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.441627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.442314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:00.442794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:00.443204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:00.486234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:00.494646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d7fb6ff0-34d0-4954-b977-59090a5addd5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.169157 Sep 18 06:31:00.495234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 555/1143] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:00 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:00.610504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:00.626886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3467aee9-bec2-47a2-b01e-407cce8115c7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.197262 Sep 18 06:31:00.627441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 589/1144] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:00 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:01.156439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:01.159296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:01.166012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.166781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.167263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.167963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.168569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.168993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.197476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e6aec1a3-4215-40cf-8801-156e798a84b1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045135 Sep 18 06:31:01.198017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 556/1145] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:01 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:01.336318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:01.338468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:01.345586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.346069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.346505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.347215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.347665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.348089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.377980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-75320d4e-7037-4f74-a45d-def602e370ba tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045655 Sep 18 06:31:01.378511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 590/1146] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:01.452671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:01.453206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:01.459122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.459605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.460037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.460722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.461168 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.461585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.518712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:01.519315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:01.525148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.525627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.526059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.526773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.527222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.527636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.633611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:01.643686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:01.645273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:01.646498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7e47902a-5ba6-479c-9dea-1f141ad2f087 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.197900 Sep 18 06:31:01.647091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 557/1147] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:01 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:01.651654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:01.654015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:01.660413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.660903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.661347 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.662048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:01.662499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:01.662957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:01.680617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:01.688264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b57d455d-c5cf-47e7-9032-44ccbacf508e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.173430 Sep 18 06:31:01.688818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 591/1148] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:01 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:01.692225 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-01b2776c-fac2-4738-8a84-73648e3846a7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044106 Sep 18 06:31:01.692782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 558/1149] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:01 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:02.210189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:02.212333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:02.218286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.218805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.219243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.219927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.220409 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.220832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.248995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-481da81f-0f8a-4c97-8726-e3b68d9027c3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042798 Sep 18 06:31:02.249537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 592/1150] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:02 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:02.390171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:02.392431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:02.398882 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.399369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.399813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.400505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.400968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.401394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.404959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:02.430870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cf902bdc-ccef-48e5-ad8a-b2f34a3c43ef tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044677 Sep 18 06:31:02.431421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 559/1151] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:02.672024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:02.672623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:02.679686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.680162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.680594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.681269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.681715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.682131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.704486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:02.706727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:02.712798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.713296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.713731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.714432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.714968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.715351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.747227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f06f3814-4398-49c3-a601-3b6dcc674a97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046610 Sep 18 06:31:02.747791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 560/1152] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:02 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:31:02.753056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:02.753656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:02.759202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.759706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.760103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.760790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:02.761233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:02.761648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:02.840916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:02.850241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:02.851723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:02.852900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-067a9eb2-67c6-4316-82a9-6309f886acb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.184822 Sep 18 06:31:02.855971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 593/1153] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:02 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:02.906953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:02.915187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-12ee6a20-0053-488f-b894-0f74ab5bd4f8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.166455 Sep 18 06:31:02.915749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 561/1154] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:02 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:03.262330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:03.264530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:03.271134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.271608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.272038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.272741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.273188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.273602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.302279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cc64fbba-7c11-4454-bed8-05ade2bd2c30 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044049 Sep 18 06:31:03.302856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 594/1155] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:03 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:03.443275 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:03.445433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:03.452417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.453075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.453853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.455428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.456120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.456558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.499506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-363cd634-a7f9-4ffe-ad1f-ada3a3171d90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059885 Sep 18 06:31:03.500626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 562/1156] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:03.760629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:03.762837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:03.769873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.770349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.770813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.771495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.771945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.772369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.801525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-848c17fd-f74c-42c4-b8f1-66153c2c9d62 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044807 Sep 18 06:31:03.802086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 595/1157] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:03 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:03.882023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:03.882699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:03.888588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.889072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.889509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.890196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.890673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.891102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.938388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:03.939687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:03.945545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.946023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.946456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:03.947168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:03.947617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:03.948910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:04.304982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:04.314325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:04.315878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:04.317155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1e109967-18c1-4e0e-a869-19caf2422a49 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.438993 Sep 18 06:31:04.317706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 563/1158] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:03 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:04.322202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:04.325171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:04.330999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:04.331493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:04.331934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:04.332634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:04.333086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:04.333510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:04.350749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:04.358208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2076dac8-9140-4606-9ec2-a490f666a7d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.423643 Sep 18 06:31:04.358800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 596/1159] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:03 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:04.364527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-75814747-8c44-4d27-9c14-fe4dc05d3ea9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045781 Sep 18 06:31:04.365082 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 564/1160] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:04 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:04.515081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:04.517260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:04.523185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:04.523668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:04.524108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:04.524807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:04.525240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:04.525664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:04.555382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c051b910-d9aa-4d66-8663-09a7774a24e8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044323 Sep 18 06:31:04.555850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 597/1161] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:04 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:31:04.813504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:04.815617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:04.822297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:04.822875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:04.823306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:04.824051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:04.824499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:04.824918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:04.855641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-390bdbed-fe92-4ed0-8dc8-690648cc31e4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046116 Sep 18 06:31:04.856181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 565/1162] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:04 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:05.342479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:05.343112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:05.349594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.350068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.350495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.351237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.351681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.352099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.377728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:05.379965 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:05.386007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.386494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.386967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.387669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.388126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.388555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.419598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a385303e-10ec-486f-bb90-c1fb34589a6b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046646 Sep 18 06:31:05.420162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 566/1163] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:05 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:31:05.425090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:05.425758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:05.431593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.432087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.432527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.433259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.433705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.434139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.512154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:05.521459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:05.522921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:05.524077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-26390cc8-c82f-4a92-bfbf-92e32626efab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.185493 Sep 18 06:31:05.524911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 598/1164] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:05 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:05.567696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:05.569885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:05.575760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.576249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.576600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:05.576735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.577423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.577877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.578304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.584962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4bd4f8f6-b5a9-40a7-aa82-d9d9c0ab6ddc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.163837 Sep 18 06:31:05.585539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 567/1165] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:05 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:05.608778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-436d2b95-95b8-449b-9cc2-d028000db838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044935 Sep 18 06:31:05.609258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 599/1166] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:31:05.868158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:05.870269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:05.877394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.877875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.878321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.879037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:05.879493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:05.879922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:05.908907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-362084ec-01fc-4fed-87a8-62b101163cc3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044872 Sep 18 06:31:05.909457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 568/1167] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:05 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:06.432648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:06.434820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:06.441878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.442354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.442811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.443484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.443928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.444346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.473734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eb9a8037-c614-4c5a-99d0-483c72722321 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044977 Sep 18 06:31:06.474286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 600/1168] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:06 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:06.549770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:06.550377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:06.556328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.556814 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.557255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.557943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.558450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.558847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.608586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:06.609194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:06.615766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.616250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.616684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.617373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.617823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.618244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.726316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:06.735332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:06.737390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:06.738566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fc69fb1c-b9f8-4b2b-a041-a128de75bd74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.193300 Sep 18 06:31:06.756139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 569/1169] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:06 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:06.760095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:06.762324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:06.768253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.768874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.769170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.769861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.770313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.770767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.771380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:06.779112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-db48d271-0f79-4734-97bb-209a231742ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.174330 Sep 18 06:31:06.779668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 601/1170] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:06 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:06.800964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-954e77e5-545b-4fa7-89d3-f7d13e185d95 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060801 Sep 18 06:31:06.801508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 570/1171] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:06.920799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:06.923015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:06.928954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.929436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.929869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.930548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:06.931025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:06.931446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:06.959884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8a403ac0-45e0-44b4-a345-0c7174ae4a26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043104 Sep 18 06:31:06.960427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 602/1172] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:06 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:07.485499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:07.487676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:07.494172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:07.494825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:07.495537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:07.496305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:07.496749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:07.497379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:07.528858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-23c776a9-2953-4031-9e5c-64a54efa88eb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047319 Sep 18 06:31:07.529412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 571/1173] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:07 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:07.762984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:07.763509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:07.769869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:07.770350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:07.770889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:07.771582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:07.772034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:07.772460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:07.803164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:07.803784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:07.809775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:07.810274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:07.810743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:07.811444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:07.811904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:07.812330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:07.952020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:07.961204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:07.962757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:07.963941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-05449c90-9321-4810-94a7-9e6a1faad025 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.204887 Sep 18 06:31:07.964497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 603/1174] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:07 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:07.980068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:07.981970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:07.984137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:07.988361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-85bd12d1-a5f0-4d27-87d1-452d92112de4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.189023 Sep 18 06:31:07.988928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 572/1175] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:07 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:07.992226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:07.992226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:07.992226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:07.992226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:07.992687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:07.993024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:07.993624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:07.995802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:08.002443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:08.002967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:08.003408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:08.004103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:08.004573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:08.004987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:08.026294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-88e40df1-926f-4b90-b04b-4adbf50a822c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060693 Sep 18 06:31:08.026894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 604/1176] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:08.034642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e47dce56-dcc4-4bf1-a89b-e0734bfe8cca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044612 Sep 18 06:31:08.035200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 573/1177] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:07 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:08.541954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:08.544144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:08.546015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:08.551227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:08.551705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:08.552138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:08.552821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:08.553271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:08.553699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:08.582535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fd595052-ab50-41ce-ad63-91527786c1da tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045039 Sep 18 06:31:08.583104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 605/1178] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:08 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:08.989407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:08.990022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:08.996553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:08.997033 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:08.997471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:08.998175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:08.998632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:08.999059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.012608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:09.013288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:09.019539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:09.020027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:09.020465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.021146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:09.021598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:09.022021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.026617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:09.188800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:09.197263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:09.198735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:09.199907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-88f43804-920d-4f54-8030-9905ccd93572 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215095 Sep 18 06:31:09.200464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 574/1179] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:09.201790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:09.205488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:09.207857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:09.210226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-332570d6-89ca-4e8e-bf71-2699176a2774 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.201509 Sep 18 06:31:09.211252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 606/1180] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:09 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:09.213613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:09.214111 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:09.214548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.215371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:09.215857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:09.216287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.217430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:09.220403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:09.226428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:09.227003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:09.227435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.228122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:09.228570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:09.228992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.242033 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:09.247084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-683cf101-5c2d-468f-9cbe-ec1e445962af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045647 Sep 18 06:31:09.247728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 575/1181] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:09.259876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-706f5649-2343-4ab3-a241-4779070d6023 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047339 Sep 18 06:31:09.260417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 607/1182] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:09 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:09.595996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:09.598085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:09.604107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:09.604582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:09.605015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.605690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:09.606136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:09.606558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:09.635709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-840a8177-47d1-4566-bd80-c07305c3cc44 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044607 Sep 18 06:31:09.636239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 576/1183] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:09 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:10.226360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:10.227010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:10.233686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.234090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:10.234515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.234657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.234756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:10.235371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.235826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.236257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.240899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.241401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.242063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.242816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.243402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.243857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.419352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:10.427840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:10.429283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:10.430443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6e4f3fde-de3b-4f88-8ba2-ce1ed2a487cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.207985 Sep 18 06:31:10.431019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 608/1184] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:10.438128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:10.438841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:10.438841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:10.441356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5dfe8b9a-558c-41fd-8097-ec01f6fed918 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.211036 Sep 18 06:31:10.442140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 577/1185] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:10 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:10.447184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:10.448981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.449565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.450119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.450415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:10.450994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.451516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.452343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.458528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.459242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.459746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.462461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.463005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.463450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.485503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2322f83f-3481-41d3-9d8f-0f208dbc9d4b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053325 Sep 18 06:31:10.485987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 609/1186] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:10.493089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6f022c1b-3015-47a9-ad16-ed0a3068b79e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049660 Sep 18 06:31:10.493632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 578/1187] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:10 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:10.647836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:10.649996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:10.655894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.656371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.656806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.657528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:10.657980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:10.658403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:10.686922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1bca9232-e248-452a-bb3a-283590592094 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042945 Sep 18 06:31:10.687466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 610/1188] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:10 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:11.456697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:11.457306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:11.464936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:11.465891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:11.466013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.466292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.466764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.467459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.467905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.468326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.472828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.473311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.473752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.474450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.474934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.475365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.660417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:11.669091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:11.669245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:11.670522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:11.671716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1eb39cc3-3b97-432c-a795-ae16e7e42a94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.219520 Sep 18 06:31:11.689462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 579/1189] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:11 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:11.689462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:11.689862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-574e9733-a563-4826-859d-2477577eca39 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.215913 Sep 18 06:31:11.689862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 611/1190] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:11 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:11.690024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:11.690117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:11.690117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:11.690269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.690269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.690269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.690269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.690597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.690597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.690764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.691110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.691554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.692250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.692701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.693141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.717986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-83d8858f-359b-4540-a0ea-8d247ed31641 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044595 Sep 18 06:31:11.718462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 580/1191] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:11.722731 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1290dfeb-1c6a-496d-b0cf-b2a2815c3419 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043973 Sep 18 06:31:11.723444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 612/1192] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:11.723805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:11.725927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:11.731545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.732020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.732450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.733130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:11.733581 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:11.734000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:11.763107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c15266df-0b68-4290-a104-c73a98d51be4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043646 Sep 18 06:31:11.763643 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 581/1193] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:11 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:12.696963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:12.697560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:12.700190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:12.700798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:12.704166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.704641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.705072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.705753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.706197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.706638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.707127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.707605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.708037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.708716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.709163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.709586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.895680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:12.904143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:12.905583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:12.906768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f6f80422-d822-4e2d-b237-ac7c81f693c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.213706 Sep 18 06:31:12.907963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 613/1194] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:12 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:12.909067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:12.912529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:12.914662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:12.917396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-85f99b60-06e6-4424-87ee-dcd6d744e486 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.220984 Sep 18 06:31:12.918157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 582/1195] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:12 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:12.920795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.921383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.921926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.922743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.923296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.923815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.925062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:12.927225 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:12.933008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.933562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.933992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.934804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.935251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.935668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.955157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5397d6c5-6fd1-4022-96c1-9df854340d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046176 Sep 18 06:31:12.955706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 614/1196] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:12.960159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:12.962379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:12.965303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3f3deb62-592f-4c3b-a033-a0823b321a1f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045944 Sep 18 06:31:12.965857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 583/1197] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:12 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:12.967979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.968453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.968882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.970245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:12.970746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:12.971164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:12.998980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-acf9ea52-8118-4ace-8147-77b819207d21 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042303 Sep 18 06:31:12.999517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 615/1198] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:12 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:13.932496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:13.933105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:13.935465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:13.940636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:13.941088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:13.941208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:13.941306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:13.941633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:13.942315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:13.942791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:13.943212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:13.947548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:13.948025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:13.948491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:13.949172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:13.949616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:13.950033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:14.132395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:14.141015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:14.143179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:14.144344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-06defc3b-766a-4b38-9065-cd81627c0280 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215783 Sep 18 06:31:14.144902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 584/1199] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:13 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:14.145371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:14.149416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:14.151577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:14.153355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4cb21deb-95eb-480b-bb56-5d2b75db91d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216632 Sep 18 06:31:14.154084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 616/1200] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:13 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:14.157402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:14.157881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:14.158313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:14.159027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:14.159477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:14.159897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:14.166526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:14.168703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:14.174556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:14.175072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:14.175511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:14.176201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:14.176655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:14.177083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:14.190767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0a608e7d-3c1e-4e35-a294-992e725d9435 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044755 Sep 18 06:31:14.191261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 585/1201] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:14.195708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:14.197779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:14.203452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:14.204559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:14.204991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:14.205669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:14.206113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:14.206531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:14.207447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bd182212-c529-45a4-8421-e06fdd887f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044656 Sep 18 06:31:14.208019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 617/1202] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:14.234701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-99d65cfe-8521-44af-be78-c882ec583617 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042442 Sep 18 06:31:14.235267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 586/1203] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:14 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:15.169237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:15.170530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:15.175744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:15.176358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:15.177039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.177519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.177954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.178664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.179139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.179589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.182838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.183319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.183755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.184436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.184889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.185317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.378749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:15.382504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:15.387525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:15.388964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:15.390132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c3a0c04a-499e-470d-a773-6710c3cecadb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.224798 Sep 18 06:31:15.390241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-73faab31-e032-4434-924e-511525e9f6c4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.218284 Sep 18 06:31:15.402771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 618/1205] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:15 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:15.402978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 587/1205] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:15 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:15.403068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:15.403181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:15.403266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:15.403343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:15.403915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.404160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.404405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.404650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.404976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.405100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.405676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.405788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.409356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.409531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.409642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.409726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.436713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-99d4bdf3-88fd-4a1b-9ca5-dcd881ab65b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044339 Sep 18 06:31:15.437084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-041828dd-3cf1-47e7-bcec-53abb5328219 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044469 Sep 18 06:31:15.437595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 619/1206] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:15.437888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 588/1207] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:15 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:15.446425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:15.448637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:15.455030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.455511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.455939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.456660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:15.457110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:15.457562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:15.486846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7cf94e73-d51b-45b1-80cb-74ce1ac60fb6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044372 Sep 18 06:31:15.487397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 620/1208] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:15 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:16.413387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:16.414006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:16.419661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:16.420265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:16.420486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.420975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.421413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.422119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.422586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.423021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.426607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.427080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.427512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.428188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.428633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.429049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.624022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:16.628083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:16.632817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5669e979-6807-4bc1-bfb2-3c334cf856be tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.223298 Sep 18 06:31:16.633378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 589/1209] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:16 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:16.639813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:16.641380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:16.644490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:16.646455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1564f60d-7605-42aa-8cb6-981eff202167 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.229906 Sep 18 06:31:16.647054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 621/1210] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:16 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:16.647562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:16.651607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:16.653713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:16.654049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.654628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.655266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.655889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.656413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.656901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.659619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.660102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.661587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.661587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.661799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.662062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.686185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-daa3e319-937b-4216-863d-22c744ff220d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051444 Sep 18 06:31:16.686651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:16.687336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 590/1211] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:16.692258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-30c2b145-d0df-419e-beb2-e3cf04657a41 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044210 Sep 18 06:31:16.692529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:16.693506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 622/1212] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:16.695364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:16.701115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.701587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.702016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.702737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.703186 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.703659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.733978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e42d1401-bbc8-48e6-8e79-0a0c95ac5af9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045425 Sep 18 06:31:16.734547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 591/1213] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:16 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:16.921392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:16.921885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:16.928601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.929078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.929515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.930238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:16.930722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:16.931145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:16.956100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:31:17.028858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-214b771c-01d0-42d8-85d8-c9ead359abc9 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 204 len: 0 microversion: 2.1 time: 0.111012 Sep 18 06:31:17.029431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 623/1214] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:31:16 2022] DELETE /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:31:17.040450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:17.041063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:17.046948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.048092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.048526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.049205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.049654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.050078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.068146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:17.180607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:17.188272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1f7e16dd-a931-4dc1-b9f0-0b5bd3d11d50 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1670 microversion: 2.1 time: 0.152711 Sep 18 06:31:17.188849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 592/1215] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:17 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1670 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:17.660139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:17.660832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:17.667342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.667821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.668254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.668941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.669388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.669807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.672490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:17.673130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:17.679843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.680325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.680759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.681456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.681907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.682331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.863768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:17.872253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e3125590-687e-4594-b226-67eed821ff90 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216805 Sep 18 06:31:17.872742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 624/1216] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:17 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:17.878101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:17.879129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:17.880270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:17.890176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:17.892995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:17.894177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5dfa766c-fbc2-4ff4-a0f6-167769980a9c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.225798 Sep 18 06:31:17.894368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.894782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 593/1217] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:17 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:17.895351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.896020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.897253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.897735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.898311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.903461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:17.904352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:17.910804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.911294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.912074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.912801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.913257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.914496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.932021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c1d333a3-640e-4323-a785-791321eb1427 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058212 Sep 18 06:31:17.932606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 625/1218] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:17 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:17.937507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:17.940431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:17.946336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.946854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.947290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.947982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:17.948432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:17.948952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:17.950041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f89c49c8-fd87-4b43-8b7c-9827b2809f40 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054199 Sep 18 06:31:17.950645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 594/1219] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:17.984697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7e155f15-6289-42e3-ada1-ee2c647acf6d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050856 Sep 18 06:31:17.985265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 626/1220] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:17 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:18.215249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:18.215879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:18.222839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:18.223598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:18.224275 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:18.225386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:18.226213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:18.226932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:18.407134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:18.414944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-741c5260-8c2e-4df2-a1bf-81a3265eb6da tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1670 microversion: 2.1 time: 0.203490 Sep 18 06:31:18.415517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 595/1221] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:18 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1670 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:18.902950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:18.903591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:18.910486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:18.911004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:18.911446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:18.912142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:18.912593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:18.913023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:18.922041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:18.922757 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:18.930993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:18.931746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:18.932363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:18.933157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:18.935479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:18.936154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.201316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:19.204629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:19.209825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-81020779-115c-4859-99e0-9db08e80c567 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.311110 Sep 18 06:31:19.210398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 627/1222] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:18 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:19.213875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:19.215333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:19.215543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:19.216825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-994144a5-40bb-464d-8a5a-92ac52fc16cf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.300138 Sep 18 06:31:19.228191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 596/1223] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:18 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:19.228191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:19.228191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:19.229535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:19.231502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.231988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.232422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.233105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.233715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.235084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.236841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.237418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.238250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.238807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.239520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.240169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.266812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-febfe322-27a3-43fd-bbed-2cf06bb45cc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047317 Sep 18 06:31:19.267389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 597/1224] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:19.272161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:19.274492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8f76f542-2f50-479b-9f2d-7a3a3d0d7c01 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062862 Sep 18 06:31:19.274672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:19.279662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 628/1225] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:19 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:19.280475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.280970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.281505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.282390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.283042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.283653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.315355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a5c786f3-5bb9-458b-8b4b-9243ecd545d7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046706 Sep 18 06:31:19.319476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 598/1226] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:19 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:19.374464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-c7770563-b45c-4cf4-bcc2-ed935f3816dc req-feea0705-cf51-40b2-a00c-97ec75bf9256 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08ce084d-f9c1-4025-a286-607e6b3407c3", "name": "network-vif-deleted", "tag": "a5589a27-a50a-4b1d-b39f-fec92bc61176"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:31:19.386051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-c7770563-b45c-4cf4-bcc2-ed935f3816dc req-feea0705-cf51-40b2-a00c-97ec75bf9256 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.386532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-c7770563-b45c-4cf4-bcc2-ed935f3816dc req-feea0705-cf51-40b2-a00c-97ec75bf9256 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.386997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-c7770563-b45c-4cf4-bcc2-ed935f3816dc req-feea0705-cf51-40b2-a00c-97ec75bf9256 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.439602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-c7770563-b45c-4cf4-bcc2-ed935f3816dc req-feea0705-cf51-40b2-a00c-97ec75bf9256 service nova] Creating event network-vif-deleted:a5589a27-a50a-4b1d-b39f-fec92bc61176 for instance 08ce084d-f9c1-4025-a286-607e6b3407c3 on hv-cinder-85535 Sep 18 06:31:19.442179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:19.442830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:19.445884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-c7770563-b45c-4cf4-bcc2-ed935f3816dc req-feea0705-cf51-40b2-a00c-97ec75bf9256 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074889 Sep 18 06:31:19.447067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 629/1227] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:31:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:19.456655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.457160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.457605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.458324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:19.458811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:19.463005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:19.533506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c945a3e6a4946559aeadc63566d29f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9cbc859294e9456eb2ca751279a17f52', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:19.543776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-885c7e2e-910e-4985-854d-b0d8014cfe5b tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 200 len: 1321 microversion: 2.1 time: 0.105964 Sep 18 06:31:19.544518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 599/1228] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:19 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 1321 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:20.238100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:20.239470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:20.241896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:20.242518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:20.247520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.248008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.248442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.249142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.249698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.249864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.250193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.250343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.250653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.251350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.251802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.252226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.469687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:20.473520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:20.483489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5b9c1aa0-059f-43ef-bfcc-ea219812f1f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.251582 Sep 18 06:31:20.484091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 630/1229] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:20 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:20.490286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:20.494988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:20.497144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:20.501580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:20.502944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f8cecf4a-e216-4580-95c5-5a79248cb5b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.264919 Sep 18 06:31:20.514610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 600/1230] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:20 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:20.514882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.514882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.514882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.514882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.515334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.515334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.515511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:20.515511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:20.522360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.522360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.522360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.522360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.522867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.522867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.549502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d7c4a0f9-c162-4d56-9a7c-2d4eb39af02d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064342 Sep 18 06:31:20.554295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 631/1231] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:20 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:20.554454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8bedce81-743b-4fe0-9781-73f61672c086 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047732 Sep 18 06:31:20.554934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:20.557196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:20.558144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 601/1232] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:20 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:20.563501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.563988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.564481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.565234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.565873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.566300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.567111 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:20.567935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:20.576349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.577020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.577457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.578911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:20.579490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:20.579981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:20.593750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 HTTP/1.1 Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:31:20.596212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:20.597798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '08ce084d-f9c1-4025-a286-607e6b3407c3'} Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:31:20.598679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:20.600068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:20.601243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:31:20.601243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2d488536-2772-4993-94de-e0beea345219 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 404 len: 111 microversion: 2.1 time: 0.034731 Sep 18 06:31:20.601243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 602/1233] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:20 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:31:20.601465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-97d8a000-d5f1-487f-bc27-b5f650b1073b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049131 Sep 18 06:31:20.602716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 632/1234] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:20 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:20.606449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-80e2df16-2f44-40d6-89f1-1722f03e5b51 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '.version_select at 0x7fe670455ca0>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:20.631458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-80e2df16-2f44-40d6-89f1-1722f03e5b51 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-615109679" status: 202 len: 0 microversion: 2.1 time: 0.028564 Sep 18 06:31:20.632028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 603/1235] 10.222.0.20 () {58 vars in 1218 bytes} [Sun Sep 18 06:31:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-615109679 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 06:31:21.252674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:21.253191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:21.259656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.260162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.260594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.261342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.261800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.262219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.276677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 HTTP/1.1 Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:31:21.278414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.280001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '08ce084d-f9c1-4025-a286-607e6b3407c3'} Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:31:21.281040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.283040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.284690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:31:21.284690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bd0e565a-312b-4dee-8a56-2b2ca058eb1c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 404 len: 111 microversion: 2.1 time: 0.031393 Sep 18 06:31:21.284690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 633/1236] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:31:21 2022] DELETE /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 06:31:21.288919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:21.289547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 08ce084d-f9c1-4025-a286-607e6b3407c3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:21.296818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.297399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.297826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.298521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.299008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.299441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.313258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 HTTP/1.1 Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:31:21.314816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.316396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '08ce084d-f9c1-4025-a286-607e6b3407c3'} Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:31:21.317147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. Sep 18 06:31:21.318428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:31:21.320137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 08ce084d-f9c1-4025-a286-607e6b3407c3 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:31:21.320137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d0948f08-67b8-416c-baf1-d6d3b80b3caf tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3" status: 404 len: 111 microversion: 2.1 time: 0.031197 Sep 18 06:31:21.320137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 604/1237] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:21 2022] GET /compute/v2.1/servers/08ce084d-f9c1-4025-a286-607e6b3407c3 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:31:21.507019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:21.507622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:21.513588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.521510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.521510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.521510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.521510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.521953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.529477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:21.530281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:21.536659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.537189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.537830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.538872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.539124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.539799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.732400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:21.735798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:21.740882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-85ada856-7547-4ab0-bb46-33e7d122d8b3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.237644 Sep 18 06:31:21.741461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 634/1238] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:21 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:21.746517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:21.748423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:21.749302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:21.749949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f620f3e5-e0e8-4b4a-b291-fb7a710acf60 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.224283 Sep 18 06:31:21.751075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 605/1239] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:21 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:21.752326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:21.756155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:21.758229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.758873 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:21.759099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.759703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.760557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.761166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.761802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.765765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.766782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.767243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.768015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.768813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.769399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.796355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b391656a-fb44-4a63-8798-c4939ee52212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051746 Sep 18 06:31:21.796971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 635/1240] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:21.801887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:21.804662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:21.809593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c79dddaa-07c9-4798-9dcb-7ce242bf72ff tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057280 Sep 18 06:31:21.810825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 606/1241] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:21 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:21.811246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.811409 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.811878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.813075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:21.813533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:21.813957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:21.853696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-75ecce28-cc06-4f61-8fcf-f007a4fccca2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055492 Sep 18 06:31:21.854392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 636/1242] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:21 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:22.768998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:22.769542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:22.775977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:22.776484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:22.776916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:22.777598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:22.778047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:22.778465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:22.780145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:22.780812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:22.787485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:22.787960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:22.788392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:22.789071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:22.789513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:22.789930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:23.010096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:23.013187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:23.018718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-146cf298-7ee1-48c7-b406-d87e5d62e707 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.254217 Sep 18 06:31:23.019337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 607/1243] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:22 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:23.023862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:23.025590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:23.026237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:23.027558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:23.029051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4a8d91df-0605-434f-bffd-0ccaa8c380e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.252717 Sep 18 06:31:23.029673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 637/1244] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:22 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:23.032563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:23.033058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:23.033497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:23.034220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:23.034708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:23.035134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:23.038632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:23.040825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:23.046750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:23.047229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:23.047660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:23.048339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:23.048812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:23.049433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:23.065071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0437a048-865b-4fb1-be50-8f917c81c020 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044762 Sep 18 06:31:23.065623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 608/1245] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:23.070290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:23.072416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:23.078046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:23.079080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:23.079229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ac030548-2e66-417c-98bf-2861fa2ae4f7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047708 Sep 18 06:31:23.079522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:23.079964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 638/1246] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:23 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:23.080209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:23.080662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:23.081086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:23.109896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7c5ec467-0621-44fc-8674-d665e0464b77 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043060 Sep 18 06:31:23.110449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 609/1247] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:23 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:24.042280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:24.042920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:24.049441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.050033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.050473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.051190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.051642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.052063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.053750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:24.054367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:24.054762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:24.061355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.061846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.062287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.063513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.063968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.064398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.251686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:24.254910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:24.259375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ebaeb51e-f74d-4fde-9f8e-85d5f6e55c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.220905 Sep 18 06:31:24.259938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 639/1248] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:24 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:24.263803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:24.264456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:24.265251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:24.266603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-68d051e0-18c7-4cb9-b9c4-98b4181d86ac tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.216747 Sep 18 06:31:24.266719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:24.267148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 610/1249] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:24 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:24.272036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:24.272993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.273532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.274024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.274801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.275151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:24.275365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.275845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.281497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.282053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.282559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.283352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.283875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.284308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.301777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:24.306124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1987f264-2817-41e4-88b4-14bf9a38174d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045139 Sep 18 06:31:24.306714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 640/1250] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:24 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:24.311192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:24.314037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:24.316641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-456e4107-2928-4625-9501-0ef61fc6a232 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048247 Sep 18 06:31:24.317204 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 611/1251] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:24 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:24.319504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.320009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.320435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.321111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:24.321553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:24.321968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:24.350327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-46408e5b-bc52-48b2-93ae-d8a900b28c3a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042638 Sep 18 06:31:24.350892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 641/1252] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:24 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:25.283244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:25.283868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:25.291023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.291509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.291949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.292649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.293109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.293536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.303188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:25.303863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:25.310706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.311183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.311615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.312831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.313377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.314149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.489037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:25.496755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1da468e7-2b55-45d1-9a5b-20e7e16baa17 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.217438 Sep 18 06:31:25.497314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 612/1253] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:25 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:25.506697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:25.506966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:25.507103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:25.510683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.511168 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.511601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.512290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.512736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.513161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.513271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:25.514801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:25.516525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-561be1e0-e530-4055-b695-11765ad2b4b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.217226 Sep 18 06:31:25.517328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 642/1254] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:25 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:25.534862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:25.537000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:25.543031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.543198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-451380ff-7c49-4625-9df3-1202f43cd48c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044629 Sep 18 06:31:25.543517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.547851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.548005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 613/1255] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:25.548118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.548118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.548118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.548686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:25.550800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:25.556424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.556908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.557337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.558017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:25.558524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:25.558975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:25.575256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5b635347-b3c2-49d9-8a7e-b4a39b74bbd3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044340 Sep 18 06:31:25.575838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 643/1256] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:25 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:25.588482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2a43bbcf-82bf-4b72-8b7e-51aac699a388 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043447 Sep 18 06:31:25.589024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 614/1257] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:25 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:26.520946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:26.521586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:26.528473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.528956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.529387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.530072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.530517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.530969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.540828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:26.541440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:26.548355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.548859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.549302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.549990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.550445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.550897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.725426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:26.732280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:26.733081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cc8596ae-c1d7-4ddb-9ed6-f96739d2d33f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.215968 Sep 18 06:31:26.733649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 644/1258] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:26.739008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:26.741142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:26.741575 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:26.743042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:26.744255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8ad52510-e689-49d5-a03b-f42fbdf5d13b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.207343 Sep 18 06:31:26.754000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 615/1259] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:26 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:26.754191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.754191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.754191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.754191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.754509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.754634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:26.754744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.754837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:26.758118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.758615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.759052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.759736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.760185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.760615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.779559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-306c8fc0-0f16-43a4-a7a3-94e26b2c4ecf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044327 Sep 18 06:31:26.780128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 645/1260] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:26.785149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:26.787275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:26.789784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-252adee0-17fb-44b2-bc4a-d2dc51db5f7b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043985 Sep 18 06:31:26.790335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 616/1261] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:26 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:26.792935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.793420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.793854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.794569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:26.795040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:26.795462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:26.824561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-83e465de-591e-4197-b4d3-fed9796f7942 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043419 Sep 18 06:31:26.825312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 646/1262] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:26 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:27.756911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:27.757524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:27.764033 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:27.764521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:27.764961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:27.766300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:27.766791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:27.767221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:27.770748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:27.771354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:27.777706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:27.778190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:27.778664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:27.779358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:27.779810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:27.780234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:27.968011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:27.973124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:27.976320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d59be41c-91e4-4262-9941-420a0aa27e84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.223227 Sep 18 06:31:27.976893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 617/1263] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:27 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:27.981581 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:27.982881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:27.983872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:27.984814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:27.985964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-09232c94-ffdd-472f-a2e0-f9f9433c49a9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.219120 Sep 18 06:31:27.986709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 647/1264] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:27 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:27.991095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:27.993284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:27.998686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:27.998941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:27.999635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:27.999803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:28.000713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:28.000822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:28.001398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:28.001518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:28.001848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:28.002089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:28.002268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:28.002822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:28.030755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0d78dab0-6f20-4537-8979-92b40a1cf88a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043066 Sep 18 06:31:28.031231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 648/1265] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:27 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:28.031988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-639c48a9-2696-4ad2-8bb5-ba6f28958136 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054065 Sep 18 06:31:28.034139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 618/1266] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:28.035957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:28.038043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:28.043592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:28.044066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:28.044493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:28.045166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:28.045620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:28.046041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:28.074932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bda2be7f-f37a-41d3-b561-586c10e2e2a7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042656 Sep 18 06:31:28.075464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 649/1267] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:28 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:29.001367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:29.002013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:29.008747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.009228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.009662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.010346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.010828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.011248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.012523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:29.013164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:29.019431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.019943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.020377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.021067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.021515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.021951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.204365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:29.207467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:29.212163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2dbf6fc3-8c8a-4b68-948e-1e9724f09cce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.215540 Sep 18 06:31:29.212819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 619/1268] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:28 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:29.216390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:29.217542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:29.217817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:29.219506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-329d78b9-b2c5-494b-892b-1362a990d436 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.210973 Sep 18 06:31:29.228063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 650/1269] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:29 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:29.228254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:29.228354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:29.228446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.228446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.228446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.228446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.228736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.228812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:29.228902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.235223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.235707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.236142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.236830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.237284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.237706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.259328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-24626aa1-8374-4ccb-8373-7077e30aa94b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045239 Sep 18 06:31:29.259828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 620/1270] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:29 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:29.264479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:29.267208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:29.267919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8d0841bf-9c3e-43e1-a9a4-e41f7728b365 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046869 Sep 18 06:31:29.268619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 651/1271] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:29.272990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.273472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.273910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.274683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:29.275144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:29.275571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:29.306686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-40d96611-9f62-434b-a345-cbd7e5dbbbe6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045707 Sep 18 06:31:29.307278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 621/1272] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:29 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:30.235624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:30.236239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:30.238780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:30.243940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:30.244448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.244668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:30.244779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.245140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.245838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.246297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.246759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.251692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.252175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.252609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.253294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.253742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.254161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.451388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:30.456353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:30.459986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dce6035f-c4f0-4adb-b7da-a4f88bc43f00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.228157 Sep 18 06:31:30.460578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 652/1273] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:30 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:30.465320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:30.467479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:30.473113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:30.473992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.474639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.474937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:30.475067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.475759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.476213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.476522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-585e9721-a3f6-4eab-8376-816acf744c78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.236465 Sep 18 06:31:30.476636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.477367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 622/1274] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:30 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:30.482074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:30.484226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:30.490172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.490690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.491129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.491816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.492274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.492692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.507076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9a733a6f-21e1-45d7-b265-4f6cd982063d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045445 Sep 18 06:31:30.507633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 653/1275] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:30 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:30.512096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:30.514195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:30.520451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.520930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.521360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.522043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:30.522495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:30.522941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:30.523056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8e20c0a3-d98d-4296-a0b9-f3561180fa56 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044538 Sep 18 06:31:30.523558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 623/1276] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:30 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:30.551030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5c514752-df11-43a6-9407-9ef617599bab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042397 Sep 18 06:31:30.551611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 654/1277] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:30 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:31.483750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:31.484289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:31.490842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.491321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.491755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.492445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.492894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.493316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.501893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:31.502522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:31.508915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.509383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.509813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.510494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.510980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.511401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.684034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:31.692514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b1f80362-fd73-401f-8a02-4918d75e4080 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.212683 Sep 18 06:31:31.693099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 624/1278] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:31 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:31.697878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:31.698189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:31.700145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:31.700554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:31.707108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.707607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.708037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.708742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.708890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:31.709197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.709676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.710268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:31.711466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e54569e-9746-44f6-a694-fbd3e75a2c2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214193 Sep 18 06:31:31.712228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 655/1279] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:31 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:31.718249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:31.720679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:31.727957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.728748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.729672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.730608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.731256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.731683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.741215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5ac88ff0-813c-4628-8547-1cf75d53d72a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047010 Sep 18 06:31:31.741894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 625/1280] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:31 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:31.746396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:31.748561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:31.756819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.756819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.756819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.757165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:31.757365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:31.757764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:31.764997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cf0c5e0f-4b00-49bd-abd2-7acb55e54695 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051496 Sep 18 06:31:31.765781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 656/1281] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:31.787823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a1de164e-22c9-4948-b2e3-079222d480c9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044997 Sep 18 06:31:31.788612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 626/1282] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:31 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:32.716100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:32.716702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:32.723321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.723807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.724245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.724929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.725381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.725803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.737138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:32.737742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:32.744076 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.744558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.745028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.745722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.746172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.746682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.915842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:32.923666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1f324638-7e93-4b1a-9f98-7d16a2309de2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.211423 Sep 18 06:31:32.924241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 657/1283] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:32 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:32.927281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:32.928779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:32.930940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:32.936404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:32.936642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.937608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.938040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.938419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:32.939069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.939536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.939696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0242546e-5c8b-4439-a8a5-7dc9b694117c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.206347 Sep 18 06:31:32.939959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.940480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 627/1284] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:32 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:32.945195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:32.947399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:32.953840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.954339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.954802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.955513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.955966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.956391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.960040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:32.970785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d358878d-d4aa-4d37-b3da-7bbce86c3d9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045408 Sep 18 06:31:32.971264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 658/1285] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:32 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:32.975703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:32.977797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:32.983292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.983800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.984224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.984902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:32.985347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:32.985622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6295aab7-fa0a-4396-9aae-357769eb23b0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044056 Sep 18 06:31:32.985753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:32.986188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 628/1286] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:32 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:33.017101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-383ca6bc-fc83-40ef-b251-bcd29f2fc570 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044843 Sep 18 06:31:33.017643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 659/1287] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:32 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:33.947045 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:33.947777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:33.954771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:33.955255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:33.955691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:33.956376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:33.956830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:33.957255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:33.965026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:33.965650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:33.972625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:33.973107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:33.973539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:33.974237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:33.974713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:33.975135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:34.404446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:34.408247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:34.412310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a2bf4f0f-476e-44cd-8e71-3a31631bc0fc tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.469260 Sep 18 06:31:34.412884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 629/1288] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:33 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:34.416958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:34.417405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:34.418414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:34.419545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:34.419674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5c06ba7a-dc2a-4c25-bfd8-f6ac94c36b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.458395 Sep 18 06:31:34.430277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 660/1289] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:33 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:34.430277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:34.430569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:34.430569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:34.430569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:34.430569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:34.431045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:34.431138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:34.431138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:34.433410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:34.433895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:34.434329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:34.435044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:34.435495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:34.435918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:34.460208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4b72f63a-14bd-4731-9219-03758a916747 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046265 Sep 18 06:31:34.460967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 630/1290] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:34 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:34.465351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:34.467514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:34.471350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-85d23ed0-1d19-4994-af9f-0ecbb3c6a886 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050189 Sep 18 06:31:34.472099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 661/1291] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:34.473613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:34.473817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:34.474254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:34.474977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:34.475427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:34.475847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:34.505465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a81ac31a-6203-44fa-9a77-d024187a4d8b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043686 Sep 18 06:31:34.506010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 631/1292] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:34 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:35.436023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:35.436632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:35.443280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.443763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.444203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.444970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.445305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:35.445711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.445793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.445967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:35.453372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.453909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.454263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.455003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.455457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.455921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.645207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:35.648535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:35.653699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0eab4f0b-6813-4e24-a426-d32421b9aa06 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.221387 Sep 18 06:31:35.654280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 662/1293] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:35 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:35.658139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:35.659710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:35.659955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:35.661027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d617942f-92a3-4cba-9fee-ded18fe46d82 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.219972 Sep 18 06:31:35.661777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 632/1294] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:35 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:35.666253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:35.668466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:35.670261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:35.675144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:35.675728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.676204 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.676723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.677338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.677974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.678223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.681488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.682014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.682488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.683303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.683801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.684272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.708535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a13ad8a5-35bc-46dd-b05b-9e759bb7e532 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045727 Sep 18 06:31:35.709241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 633/1295] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:35.713503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:35.715626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:35.715935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a7984940-fbf2-4aba-806b-1a9ab8f50759 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060594 Sep 18 06:31:35.716538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 663/1296] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:35 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:35.721835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.722318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.722786 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.723493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:35.723928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:35.724350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:35.753521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d1c0b519-d8e6-4896-8c89-84cd0d8edece tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043547 Sep 18 06:31:35.754069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 634/1297] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:35 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:36.678179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:36.678811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:36.685175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.685652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.686104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.686794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.687243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.687663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.688484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:36.689101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:36.695415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.695896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.696369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.697055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.697509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.697933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.884189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:36.887939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:36.893173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9c23c082-5b9e-4629-8f03-29c9397b9e1e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.218828 Sep 18 06:31:36.893762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 664/1298] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:36 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:36.897489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:36.898476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:36.899145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:36.900346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9367d8d6-e729-4a09-ad7d-34e28dd53800 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215956 Sep 18 06:31:36.901046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:36.901162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 635/1299] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:36 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:36.911404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:36.911569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.911569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.911769 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:36.911851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.911851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.911851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.911851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.916258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.916749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.917188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.917919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.918376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.918844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.939936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9d9b0a36-7a04-4010-986d-2cd6f382f1f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045135 Sep 18 06:31:36.940512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 665/1300] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:36.945110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:36.947893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:36.951187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0fd8a67b-0ca2-42fa-b133-001fd337f07f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049215 Sep 18 06:31:36.951757 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 636/1301] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:36 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:36.956325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.957142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.957776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.958676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:36.959139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:36.959572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:36.993561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d430bfe5-7230-43b6-b9b4-90a7727d0bcd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052030 Sep 18 06:31:36.994114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 666/1302] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:36 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:37.919115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:37.919756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:37.925617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:37.926216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:37.926808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:37.927299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:37.927735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:37.928431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:37.928887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:37.929315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:37.932518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:37.932996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:37.933433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:37.934117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:37.934564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:37.935006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:38.122480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:38.125856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:38.130947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cac3f551-5983-4b81-a306-531cde8dfd74 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.215621 Sep 18 06:31:38.131526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 637/1303] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:37 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:38.134680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:38.136196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:38.137297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:38.137429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-18f3d0a3-1977-4d21-a520-6366a23f697c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215635 Sep 18 06:31:38.138215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 667/1304] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:37 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:38.139720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:38.146237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:38.146764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:38.147208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:38.147919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:38.148364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:38.148794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:38.152054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:38.154419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:38.160345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:38.160824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:38.161263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:38.161947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:38.162399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:38.162854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:38.180080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-aec64d42-67e9-4f0b-953a-a9fd5e6443ed tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046740 Sep 18 06:31:38.180640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 638/1305] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:38 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:38.185348 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:38.187967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:38.193555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:38.193705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bceea86b-3d48-481d-9174-efa6f3e23a28 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045444 Sep 18 06:31:38.194031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:38.194462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:38.194619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 668/1306] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:38 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:38.195170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:38.195614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:38.196034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:38.225076 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-549d2ba6-6cfb-4405-97d3-80a49075d5dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043423 Sep 18 06:31:38.225614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 639/1307] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:38 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:39.154593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:39.155175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:39.161791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.162368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.162838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.163028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:39.163520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.163623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:39.163966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.164385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.169936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.170418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.171406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.172094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.172548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.172971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.358810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:39.362035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:39.366600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e186475-8fca-491a-a947-956694112329 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.215823 Sep 18 06:31:39.367186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 669/1308] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:39 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:39.370800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:39.371810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:39.372304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:39.373471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d0830f44-802f-408c-b8aa-0c72d64d46bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214271 Sep 18 06:31:39.373919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:39.380435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.380975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.381415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.382116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.382571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.383017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.383230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 640/1309] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:39 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:39.388087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:39.390271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:39.396118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.396593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.397026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.397716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.398181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.398594 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.412910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fbd94a95-99cb-489f-b6a6-75d0ceeb8bda tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044630 Sep 18 06:31:39.413404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 670/1310] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:39 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:39.419035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:39.421638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:39.427348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.427661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d15a5047-ee98-4564-93ad-0513c87053bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043357 Sep 18 06:31:39.427829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.428219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 641/1311] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:39 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:39.428355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.428946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:39.429402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:39.429824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:39.459250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d59e2b1a-81d8-4361-b8a7-5ca541f430c7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044822 Sep 18 06:31:39.459809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 671/1312] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:39 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:40.390503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:40.391176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:40.397799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.398286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.398745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.399328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:40.399776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.399877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.399978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:40.400272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.406909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.407383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.407811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.408494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.408940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.409361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.424631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:40.594900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:40.597969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:40.604073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7d2b1139-95b1-41cd-848d-ea9808a74113 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.217371 Sep 18 06:31:40.604637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 642/1313] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:40 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:40.606469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:40.607955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:40.609665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3c74d901-e1e1-4fd3-8fcb-9e6fd08732dc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214176 Sep 18 06:31:40.609775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:40.610408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 672/1314] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:40 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:40.611914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:40.618071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.618556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.619052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.619738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.620189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.620608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.626354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:40.628568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:40.634484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.635043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.635544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.636300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.636846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.637335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.651019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-228c5924-c586-467a-8d3e-c5654f1ff401 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045342 Sep 18 06:31:40.651759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 643/1315] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:40.656813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:40.658966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:40.664537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.664978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.665410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.666093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:40.666542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:40.667008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d34135bd-5853-4f1e-b0bd-b3bb91b14702 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051496 Sep 18 06:31:40.667118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:40.667570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 673/1316] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:40 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:40.691262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:40.696927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cfdf227c-94eb-4e4d-b7ef-edebdfae96ac tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044320 Sep 18 06:31:40.697470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 644/1317] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:40 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:41.627640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:41.628250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:41.636137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.636541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.636717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:41.637176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.637280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:41.637763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.638224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.638673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.644454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.644941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.645379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.646077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.646528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.646976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.833106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:41.836984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:41.841054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7f12f569-5ea9-44c6-9c51-c0c1dd9893f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.217851 Sep 18 06:31:41.841624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 674/1318] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:41 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:41.846289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:41.847835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:41.849051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f6e1617d-f0c1-4d0c-b60c-300591ff1299 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.216521 Sep 18 06:31:41.863717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 645/1319] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:41 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:41.863935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:41.863935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:41.864096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:41.864096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:41.864237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.864237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.864237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.864237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.864535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.864535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.864692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.864692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.864692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.864692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.864980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.865840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.891396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-40907171-a450-421e-a1f0-1eb686c6290a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048732 Sep 18 06:31:41.891964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 675/1320] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:41.895006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2a16c9c2-c8f4-4059-9ac8-b4900f98612c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044309 Sep 18 06:31:41.895644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 646/1321] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:41 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:41.896758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:41.898901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:41.904508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.905484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.905936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.906624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:41.907076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:41.907499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:41.937532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ec1b1d01-82ab-4d9b-9d33-1f5b014a9875 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044565 Sep 18 06:31:41.938073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 676/1322] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:41 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:42.864239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:42.864853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:42.871396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:42.871878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:42.872312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:42.873258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:42.873448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:42.874548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:42.875168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:42.875780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:42.882361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:42.882867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:42.883307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:42.884009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:42.884461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:42.884889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:43.081620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:43.085263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:43.091679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:43.093107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-31c5dd96-824e-4abd-a910-f06f76bfe65c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.232783 Sep 18 06:31:43.093311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:43.094747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0a9fe405-49d3-4e9c-bd34-0583d055f112 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.223958 Sep 18 06:31:43.095622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 677/1323] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:42 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:43.102946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 647/1324] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:42 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:43.103487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:43.104082 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:43.105825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:43.106863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:43.113509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:43.113999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:43.114482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:43.115325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:43.115456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:43.115784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:43.116029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:43.116214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:43.116467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:43.117169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:43.117624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:43.118080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:43.149807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-36fc44ce-1040-40e1-824e-45330ce0a536 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052863 Sep 18 06:31:43.150607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 678/1325] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:43.150736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-85eedb7b-7104-49c6-9565-62cff91f8562 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050778 Sep 18 06:31:43.154075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 648/1326] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:43 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:43.155176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:43.157283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:43.163038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:43.163515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:43.163945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:43.164659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:43.165109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:43.165531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:43.195388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-27eb20c9-9195-44e8-a566-8da470a2bc1d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043952 Sep 18 06:31:43.196147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 679/1327] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:43 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:44.116612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:44.117227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:44.120004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:44.120620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:44.124567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.125048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.125482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.126173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.126652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.127081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.128733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.129233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.129658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.130344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.130827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.131253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.322395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:44.326312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:44.331349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-65fbd847-ccbf-489e-811d-deeb16445f9f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.218553 Sep 18 06:31:44.332196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 649/1328] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:44 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:44.337727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:44.340158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:44.346271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.346804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.347241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.347936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.348387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.348809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.351510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:44.352977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:44.354161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1be2552a-b665-4d43-8046-5c7adc8305ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.238149 Sep 18 06:31:44.354748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 680/1329] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:44 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:44.359848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:44.362002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:44.367880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.370208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.371021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.371735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.372194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.372620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.381893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e3e9e4b8-9515-4836-94c0-7ba3d15152b7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048779 Sep 18 06:31:44.386947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 650/1330] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:44.387647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:44.389944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:44.396798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.397411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.397947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.398774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:44.399307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:44.399730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:44.404634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-30b41237-825e-4aaa-b8f8-50bfdff0a9d7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048255 Sep 18 06:31:44.405438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 681/1331] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:44 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:44.430385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8113bb5b-79b0-438c-8de0-f3d4c327d107 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046594 Sep 18 06:31:44.430968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 651/1332] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:44 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:45.358542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:45.359175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:45.365615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.366098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.366534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.367254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.367710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.368138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.380580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:45.381190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:45.387507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.388023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.388462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.389827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.390179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.390623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.566222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:45.574206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a361515e-6e69-4a08-a533-3a5a8bbaf0b9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.219528 Sep 18 06:31:45.590650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 682/1333] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:45 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:45.590866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:45.590989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:45.590989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:45.591346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:45.591448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.591530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:45.591627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.591627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.591779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1d86b150-9a22-403d-b46c-1169560a472a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.213145 Sep 18 06:31:45.591856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.591856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.592020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 652/1334] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:45 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:45.592104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.594968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:45.597105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:45.602775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.603260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.603751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.604460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.604891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.605312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.620559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-62359e75-25fb-4a12-a39b-6b0d7e8b896f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044433 Sep 18 06:31:45.621276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 683/1335] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:45 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:45.629004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:45.631181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:45.634518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-68904ec7-b79e-4dad-82f7-1619115116fa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043072 Sep 18 06:31:45.635109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 653/1336] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:45 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:45.636928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.637407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.637841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.638530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:45.639009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:45.640039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:45.669505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-27ba57dd-dd44-416c-86cd-a664d8768f90 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044210 Sep 18 06:31:45.670089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 684/1337] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:45 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:46.597928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:46.598540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:46.604936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.605414 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.605846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.606525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.607005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.607427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.614900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:46.615537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:46.621945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.622422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.622897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.623612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.624320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.624742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.807802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:46.811815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:46.815564 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0881631d-7c09-4a68-99b9-7678ff132672 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.221450 Sep 18 06:31:46.816351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 654/1338] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:46 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:46.820741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:46.822144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:46.822274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:46.823472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c2ff9926-c066-4712-a4a4-1425bcd5db5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.212488 Sep 18 06:31:46.824185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 685/1339] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:46 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:46.824792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:46.831913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.832411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.832970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.833667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.834122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.834548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.836067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:46.838181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:46.844654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.845129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.845560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.846247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.846740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.847165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.865807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fd40eec2-91ef-4672-a285-35e7f6d0cde4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047745 Sep 18 06:31:46.866527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 655/1340] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:46.871388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:46.873506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:46.876309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7300f725-90d4-42f9-a126-3ecc2891ff2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043857 Sep 18 06:31:46.877024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 686/1341] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:46 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:46.879978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.880459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.880887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.881569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:46.882021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:46.882441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:46.913406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-09e6b755-bd32-49b3-afb3-60a4cbe58add tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045740 Sep 18 06:31:46.914078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 656/1342] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:46 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:47.838215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:47.838828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:47.845238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:47.845713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:47.846145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:47.846857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:47.847304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:47.847721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:47.848740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:47.849352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:47.855523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:47.856011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:47.856454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:47.858006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:47.858683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:47.859121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:48.053436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:48.056849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:48.063280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a24f1b00-bb96-46da-ae31-f806b745ecc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.228987 Sep 18 06:31:48.063901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 687/1343] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:47 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:48.066135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:48.067614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:48.068794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0fa708f1-24a7-4d97-a486-c6661224c4d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.224128 Sep 18 06:31:48.068917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:48.069496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 657/1344] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:47 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:48.071003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:48.073871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:48.076109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:48.077831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:48.078327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:48.078792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:48.079499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:48.079956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:48.080389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:48.083011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:48.083503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:48.083946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:48.084638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:48.085094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:48.085545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:48.110540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-373b90b6-99d2-459e-9e48-2ec3bd600a1e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045510 Sep 18 06:31:48.111054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 688/1345] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:48.115601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:48.117384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1b96d69e-bb40-4955-8082-fadf80273cbb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047005 Sep 18 06:31:48.117709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:48.118127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 658/1346] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:48 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:48.124257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:48.124739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:48.125172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:48.125859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:48.126308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:48.126760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:48.130315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:48.155343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-728399c9-3e31-4c38-a342-f29be8d9d93d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043310 Sep 18 06:31:48.155896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 689/1347] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:48 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:49.097543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:49.098155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:49.104248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.104732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.104847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:49.105248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.105445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:49.105847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.106301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.106765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.111552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.112040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.112478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.113171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.113622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.114046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.299137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:49.302277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:49.307404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-afb54982-7c05-416a-a217-e4575f1012e5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.213907 Sep 18 06:31:49.307996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 659/1348] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:49 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:49.313084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:49.315268 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:49.318820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:49.320356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:49.320943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.321430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.321557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-741ba294-a74a-4ad9-bd36-90ab38236cf9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.220618 Sep 18 06:31:49.321867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.325473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 690/1349] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:49 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:49.325636 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.325636 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.325636 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.329389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:49.330303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:49.336224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.336707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.337145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.337836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.338292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.338744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.359548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3b011af4-8e8c-4f42-99e7-c3c78078d34f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050356 Sep 18 06:31:49.360016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 660/1350] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:49.364702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:49.366889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:49.372046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-58762e06-1b00-4e88-87a4-be22c646737c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046540 Sep 18 06:31:49.372046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 691/1351] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:49 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:49.372766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.373992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.374432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.375160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:49.375610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:49.376035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:49.400318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:49.417520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ce1f9919-9d4a-4eb9-9e27-690586a47ced tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056402 Sep 18 06:31:49.418503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 661/1352] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:49 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:50.331102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:50.331756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:50.338310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.338834 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.339275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.339963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.340419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.340843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.348707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:50.349320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:50.355664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.356151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.356589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.357280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.358420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.358895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.540371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:50.544231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:50.548167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dae512e2-cdfd-4ef1-b05b-1e32cbd2519b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.221010 Sep 18 06:31:50.548733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 692/1353] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:50 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:50.553296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:50.553445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:50.554794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:50.555685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:50.556120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8ba66018-018d-4fbc-8c61-ad8b9071a335 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.211233 Sep 18 06:31:50.556812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 662/1354] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:50 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:50.561382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:50.563506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:50.568244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.568728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.569159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.569312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.569746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.570185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.570312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.570656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.570899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.571263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.571395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.571765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.601201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-672f6766-5f77-4442-a99d-e3b4a3d0f32a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051358 Sep 18 06:31:50.601606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cd6a6ae0-e82e-4893-8801-a9a93000cbc4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043866 Sep 18 06:31:50.601729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 693/1355] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:50.606138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 663/1356] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:50 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:50.606288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:50.608345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:50.613867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.614338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.614799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.615472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:50.615917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:50.616337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:50.646236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b28dad78-3533-4e27-8569-5d2f6a8054b4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043620 Sep 18 06:31:50.646818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 694/1357] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:50 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:51.580735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:51.581345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:51.587685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.588159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.588587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.589264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.589708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.590125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.617426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:51.618096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:51.624396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.624878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.625312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.625996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.626441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.626898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.777109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:51.786260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:51.787730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:51.788908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0b8a96c-bb69-4669-a413-04fe1f37bb7a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.212062 Sep 18 06:31:51.789467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 695/1358] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:51 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:51.794097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:51.796905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:51.803245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.803734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.804170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.804293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:51.804868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.805321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.805744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.812240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dc615d75-7bbb-437c-bc29-c697c34a6854 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.245266 Sep 18 06:31:51.813054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 664/1359] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:51 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:51.817565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:51.820196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:51.825911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.826426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.826908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.827596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.828209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.828619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.835415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f4a28adc-d70a-4c47-8f1e-a015a6465df4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044864 Sep 18 06:31:51.835980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 696/1360] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:51.840580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:51.843228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:51.848915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.849399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.849843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.850532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:51.851010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:51.851435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:51.861131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-45d62c27-4db1-424b-a72d-b104fa687654 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047152 Sep 18 06:31:51.861696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 665/1361] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:51 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:51.881417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-601003a8-624b-4b2a-ace4-62095e6e4bee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044441 Sep 18 06:31:51.882283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 697/1362] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:51 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:52.814810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:52.815345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:52.821659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:52.822134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:52.822567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:52.823266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:52.823714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:52.824135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:52.835915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:52.836524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:52.843083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:52.843568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:52.844005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:52.844700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:52.845149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:52.845579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:53.016659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:53.025255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:53.026713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:53.027888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9976f3db-2277-4b06-bd8e-b8fe399cd1a6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.217739 Sep 18 06:31:53.028445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 666/1363] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:52 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:53.028967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:53.033562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:53.035798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:53.037538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e959d600-915d-4b10-9b1b-e4252ce07f79 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.205384 Sep 18 06:31:53.038295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 698/1364] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:52 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:53.042013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:53.042497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:53.042849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:53.042975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:53.043632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:53.044083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:53.044503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:53.045618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:53.051545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:53.052020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:53.052450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:53.053143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:53.053579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:53.053996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:53.074770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ab380bcc-ca6a-41b1-bb70-a9ca34beae81 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045204 Sep 18 06:31:53.075262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 667/1365] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:53.080032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:53.082283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:53.087293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aa35e423-7130-4e8e-99c7-1882e20a56ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048152 Sep 18 06:31:53.087854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 699/1366] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:53 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:53.088081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:53.088573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:53.089096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:53.089865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:53.090323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:53.090784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:53.120526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b9b973c7-d94b-4323-ab34-95429ea46600 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044287 Sep 18 06:31:53.121077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 668/1367] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:53 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:54.067594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:54.068181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:54.070154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:54.070797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:54.074768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.075254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.075692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.076432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.076886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.076997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.077311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.077452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.077884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.078571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.079046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.079465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.263402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:54.272310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:54.272752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:54.274208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:54.275407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6fe84e2b-3a91-4501-b0ad-95536fb57e2d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.211774 Sep 18 06:31:54.275964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 700/1368] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:54 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:54.281054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:54.281441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b8d633ac-c618-4a6d-b907-e6005193d8d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.214751 Sep 18 06:31:54.282312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 669/1369] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:54 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:54.286960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:54.288337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:54.289291 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:54.295547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.295716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.296109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.296251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.296548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.296672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.297253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.297421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.297937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.298039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.298352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.298472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.327426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9033b520-82de-4462-9f95-07bf8fcac52f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044013 Sep 18 06:31:54.328186 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 670/1370] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:54 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:54.331090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0c237588-5983-4466-93ca-0db5f8ded140 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054031 Sep 18 06:31:54.331782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 701/1371] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:54.333220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:54.335087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:54.340756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.341229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.341659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.343049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:54.343513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:54.343945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:54.372135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1e90cc36-8a9d-4143-8f38-5010fdc64c69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042936 Sep 18 06:31:54.372677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 671/1372] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:54 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:55.301105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:55.301733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:55.304103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:55.305047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:55.306149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:55.309391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.309914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.310394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.311128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.311827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.312003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.313001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.313500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.313984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.314892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.315348 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.315771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.508841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:55.513578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:55.517700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:55.519282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:55.520452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fcf75cc0-dc85-42de-ba49-61044bb74343 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.223297 Sep 18 06:31:55.521018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 702/1373] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:55 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:55.521361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9696d4a0-0d2b-4330-8357-6d6d8c3b916c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.220224 Sep 18 06:31:55.521924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 672/1374] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:55 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:31:55.525471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:55.527608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:55.533719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.534198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.534911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.535827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.536495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.537150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.537563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:55.539788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:55.546540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.547070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.547543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.548242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.548702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.549136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.566622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34743594-e317-4da9-9982-0d38b18ac9c6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044478 Sep 18 06:31:55.567100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 703/1375] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:55 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:55.571590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:55.573709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:55.578667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-684bc577-97ac-4354-9e9c-e33b6a28bc0d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055438 Sep 18 06:31:55.579243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 673/1376] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:55.579453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.579870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.580309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.580994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:55.581444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:55.581873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:55.611418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-735bc6d7-71f3-441f-8d3f-9595c7654232 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043337 Sep 18 06:31:55.611971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 704/1377] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:55 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:56.545421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:56.546033 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:56.546512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:56.547157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:56.552498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.552979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.553412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.553521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.553930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.554105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.554373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.554557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.555006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.555102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.555530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.555956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.748915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:56.752722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:56.754988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:56.757344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-38beaf27-63ee-4bfe-9915-bb0f99948f9b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216237 Sep 18 06:31:56.757922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 674/1378] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:56 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:56.762479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:56.763047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:56.763966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:56.765133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4f86d47b-ea7f-474f-b676-4d8bb7355243 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.223011 Sep 18 06:31:56.765454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:56.765683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 705/1379] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:56 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:56.770443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:56.771675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.772220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.772716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.773465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:56.773597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.773913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.774336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.781021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.781506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.781942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.782660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.783119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.783542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.804295 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-621110e6-2e8a-4ef4-8ad1-1f74a6b9febd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045344 Sep 18 06:31:56.804859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 675/1380] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:56 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:56.809365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:56.811539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:56.813440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e4e59eed-6fd1-42b6-838b-0f81597ed386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046529 Sep 18 06:31:56.814177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 706/1381] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:56 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:56.817181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.817658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.818091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.818821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:56.819270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:56.819690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:56.848668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-04a8384d-ec6a-4cbe-8f61-ccbd3a38d7cc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042833 Sep 18 06:31:56.849199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 676/1382] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:56 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:57.781078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:57.781643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:57.788056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:57.788561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:57.788993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:57.789675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:57.790153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:57.790356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:57.790797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:57.791005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:57.797240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:57.797723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:57.798160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:57.798873 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:57.799333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:57.799756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:57.981688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:57.986282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:57.990056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-88e1a067-cffc-4c33-ac29-a27882f1e93e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.212835 Sep 18 06:31:57.990647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 707/1383] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:57 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:57.995048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:57.995204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:57.996562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:57.997325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:57.998276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-af6173c1-a24a-47bd-a7b3-b09d8742245d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.211219 Sep 18 06:31:57.999379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 677/1384] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:57 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:31:58.003891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:58.004094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:58.004784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:58.005225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:58.005917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:58.006023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:58.006371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:58.006823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:58.011648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:58.012135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:58.012574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:58.013269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:58.013734 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:58.014344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:58.018142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:58.035794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fcc781d6-50a9-42ad-9bdd-2a0e2c9623a6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044028 Sep 18 06:31:58.036272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 708/1385] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:57 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:58.040842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:58.043071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:58.045155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-19fa3056-1850-42a7-baf6-777c01a553cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044200 Sep 18 06:31:58.045726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 678/1386] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:58.051637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:58.052137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:58.052577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:58.053278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:58.053733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:58.054159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:58.084154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3ad4e7c3-5a20-4f48-9d0b-a065aa6348df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046866 Sep 18 06:31:58.084710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 709/1387] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:58 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:59.025903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:59.026570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:59.031526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:59.032263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:59.033123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.033611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.034045 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.034770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.035221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.035642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.038675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.039158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.039594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.040284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.040741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.041166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.230881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:59.233876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:59.238612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5bccc2d4-d746-4d1c-9f76-6be562c19338 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216553 Sep 18 06:31:59.239177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 679/1388] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:31:59 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:31:59.242734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:59.243677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:59.244205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:31:59.245376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c3c910e5-531a-4174-8f73-bfbaf5edaec3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.217678 Sep 18 06:31:59.245791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:59.247436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 710/1389] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:31:59 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:31:59.252362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.252843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.253274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.253970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.254403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.254847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.255391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:59.257572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:59.264416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.264945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.265378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.266066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.266520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.266980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.284495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a3ea9296-be30-4561-97ca-cd3d963b9b0d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044196 Sep 18 06:31:59.289165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 680/1390] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:59 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:59.294146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:31:59.296445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:31:59.299854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a28d0b67-4b8f-4094-934d-eb57d4659f60 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050186 Sep 18 06:31:59.300530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 711/1391] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:59 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:31:59.302282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.302847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.303272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.303957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:31:59.304405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:31:59.304825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:31:59.334354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1541d6ed-36b4-4000-b35f-e5909d5d1cc3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044190 Sep 18 06:31:59.334938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 681/1392] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:31:59 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:00.261961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:00.262626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:00.269043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.269526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.269962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.270669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.270826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:00.271227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.271432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:00.271547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.277732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.278208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.278660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.279339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.279831 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.280252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.466313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:00.471129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:00.474721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6e9a3577-c68f-4bbd-861e-130de223e70b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216574 Sep 18 06:32:00.475286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 712/1393] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:00 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:00.479877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:00.480039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:00.481382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:00.482109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:00.482597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bff94d9b-22df-4955-ba87-f6b3e29c6e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215589 Sep 18 06:32:00.490948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 682/1394] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:00 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:00.490948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:00.490948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:00.493971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.494456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.494920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.495398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:00.495606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.496053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.496503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.497262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.497754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.498192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.499203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.499801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.500088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.527562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4ad9667b-14c3-47cb-8d9e-6915e0d09ffc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051152 Sep 18 06:32:00.528043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 713/1395] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:00 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:00.531806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c9f680eb-27e6-4c9f-beed-c763e354d74d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047062 Sep 18 06:32:00.532585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 683/1396] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:00.535334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:00.537733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:00.543597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.544077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.544515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.545859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:00.546319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:00.546781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:00.575190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5421d4a3-3af7-4dd3-82c1-757f6668ce60 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046179 Sep 18 06:32:00.575737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 714/1397] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:00 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:01.498292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:01.499686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:01.506103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.506642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.507041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.507729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.508177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.508644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.511907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:01.512513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:01.519635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.520109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.520536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.521223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.521667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.522087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.710630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:01.714766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:01.718630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-635f0896-e0cf-476d-b766-9f03f1d664df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.224066 Sep 18 06:32:01.719148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 684/1398] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:01 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:01.723736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:01.723884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:01.725294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:01.726011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:01.726455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a759f965-2d75-4d9c-8395-ce285c8769c9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.218477 Sep 18 06:32:01.727167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 715/1399] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:01 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:01.731819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:01.734383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:01.740091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.740569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.741005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.741688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.742139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.742562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.744054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.744646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.745086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.745787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.746240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.746696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.772124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0a6a3f7b-1784-44f0-9963-84daeef04931 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043804 Sep 18 06:32:01.772868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 716/1400] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:01.775941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dd18f19d-b5f8-41d9-8f6a-1ac59e03cfc7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055776 Sep 18 06:32:01.776502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 685/1401] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:01 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:01.777391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:01.779522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:01.785194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.785667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.786105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.786805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:01.787251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:01.787670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:01.816690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b5cbd0a3-efb9-481a-9cb0-1b8deff65621 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042981 Sep 18 06:32:01.817235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 717/1402] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:01 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:02.742182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:02.742746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:02.749895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:02.750381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:02.750844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:02.751538 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:02.751991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:02.752416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:02.752515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:02.753094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:02.759418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:02.759890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:02.760318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:02.761000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:02.761449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:02.762024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:02.946013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:02.949804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:02.954604 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b5d98efe-2194-41be-b8f0-de9a6de6d095 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216311 Sep 18 06:32:02.955183 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 686/1403] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:02 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:02.959338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:02.960298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:02.960777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:02.961942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5742787f-6335-4eea-8b44-fce0374b6ff0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214132 Sep 18 06:32:02.962660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 718/1404] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:02 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:02.962785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:02.967172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:02.969267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:02.975050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:02.975533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:02.975838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:02.976095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:02.976319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:02.976753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:02.976870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:02.977220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:02.977440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:02.977640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:02.977890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:02.978308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:03.007472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f789e995-7d42-4866-9481-75850b713666 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043811 Sep 18 06:32:03.007789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1c5831ea-d2d2-48b0-9b13-5538946eb67a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051600 Sep 18 06:32:03.007936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 719/1405] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:02 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:03.008500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 687/1406] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:03.012696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:03.015380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:03.020948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:03.021426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:03.021875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:03.022535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:03.023009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:03.023430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:03.052547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b06bb6be-461c-4a6a-9e6c-0f7f0d41196e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043534 Sep 18 06:32:03.053085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 720/1407] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:03 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:03.976799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:03.977414 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:03.983967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:03.984447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:03.984882 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:03.985572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:03.986027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:03.986451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:03.987437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:03.988047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:03.994531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:03.995111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:03.995549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:03.996234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:03.996684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:03.997105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:04.181023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:04.189096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:04.189896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-23609c46-87ce-4edd-8564-066599d138df tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216686 Sep 18 06:32:04.190478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 688/1408] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:03 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:04.195839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:04.197984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:04.198105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:04.199550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:04.201007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2d005749-c647-453b-8c55-b5dc42fee391 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.217392 Sep 18 06:32:04.201810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 721/1409] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:03 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:04.205032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:04.205520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:04.205957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:04.206728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:04.207346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:04.207784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:04.209070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:04.211475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:04.217311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:04.217861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:04.218366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:04.219174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:04.219625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:04.220047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:04.233770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:04.239683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-18850714-3be2-4189-b98f-f33ed40fd2ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047969 Sep 18 06:32:04.240247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 689/1410] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:04 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:04.244910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:04.247050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:04.250526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e2ef4524-6482-4977-a888-2cfde51c683e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046803 Sep 18 06:32:04.251103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 722/1411] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:04 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:04.253470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:04.253944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:04.254375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:04.255098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:04.255542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:04.255955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:04.284720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-990eb1cb-3810-4451-8c80-23b1db2085ec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043381 Sep 18 06:32:04.285256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 690/1412] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:04 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:05.217387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:05.218503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:05.225034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.225513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.225943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.226658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.226772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:05.227166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.227268 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:05.227508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.234157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.234660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.235099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.235792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.236246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.236674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.424533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:05.428856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:05.432617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-69915dc4-f0f2-4c3f-9d6b-c7855e761e14 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.218996 Sep 18 06:32:05.433212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 723/1413] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:05 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:05.437924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:05.438766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:05.440100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:05.440318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:05.441726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-02e36884-0c6d-4d61-8745-077cece1a5a2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.218891 Sep 18 06:32:05.442558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 691/1414] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:05 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:05.446137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.446780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.447084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.447784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.448244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.448669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.456079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:05.458301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:05.464389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.464960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.465456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.466394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.467386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.467386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.488022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1fc5f2ec-1307-4fe2-972e-8fa4148869d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053681 Sep 18 06:32:05.488504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 724/1415] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:05.493030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:05.495178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:05.499359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ce5ec6e9-1f41-4781-bb08-fdfa8feb9ab0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047554 Sep 18 06:32:05.499914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 692/1416] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:05 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:05.501114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.501594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.502031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.502748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:05.503201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:05.503624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:05.533365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f898be0d-ddf5-4b51-bd46-7330424559a8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043869 Sep 18 06:32:05.533922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 725/1417] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:05 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:06.456385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:06.457110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:06.463823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.464386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.464824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.465520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.465969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.466388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.469820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:06.470493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:06.476896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.477375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.477810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.478502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.478994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.479417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.911486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:06.915464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:06.919311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bf2d4e6c-add8-4471-b62a-514a7603726f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.466886 Sep 18 06:32:06.919886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 693/1418] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:06 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:06.924095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:06.924449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:06.925541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:06.926603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:06.926727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2e43bf3b-4407-4417-ace8-35a19dcf99d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.461344 Sep 18 06:32:06.927468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 726/1419] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:06 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:06.932385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.932870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.933305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.933996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.934450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.934901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.942379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:06.944644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:06.950330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.950850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.951286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.951972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.952422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.953339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.960066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:06.965323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6238cda8-681d-4b52-92f5-eff40790c237 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044477 Sep 18 06:32:06.965887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 694/1420] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:06.971367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:06.973498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:06.979304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.979790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.980227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.980923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:06.981371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:06.981797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:06.982726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e82152f4-29d3-45f4-89ef-40e5ae0817da tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044165 Sep 18 06:32:06.983286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 727/1421] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:06 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:07.011005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fa025dc7-e050-4f0a-be46-39eab802da5f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044028 Sep 18 06:32:07.011619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 695/1422] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:06 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:07.942997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:07.943612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:07.950279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:07.950807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:07.951187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:07.951607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:07.951820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:07.951920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:07.952396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:07.952831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:07.958266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:07.958799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:07.959236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:07.959916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:07.960366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:07.960785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:08.145472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:08.149600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:08.153555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34df0287-1f73-45a0-8f09-d8f737e050f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.214402 Sep 18 06:32:08.154129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 728/1423] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:07 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:08.166708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:08.166876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:08.166967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:08.166967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a5fe88fa-4c48-4097-bb96-0e712d39d871 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.214049 Sep 18 06:32:08.167138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:08.167216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 696/1424] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:07 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:08.167216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:08.167368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:08.170521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:08.170521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:08.170755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:08.170839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:08.170839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:08.170839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:08.174955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:08.175437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:08.175870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:08.176557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:08.177007 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:08.177428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:08.199258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a3f127c8-6717-4bfe-97f7-4732fb752890 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044000 Sep 18 06:32:08.199749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 729/1425] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:08.205840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-eefeb6ac-230e-4096-9f01-97dd31a35db4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042971 Sep 18 06:32:08.206388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 697/1426] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:08 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:08.207438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:08.209597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:08.215385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:08.215864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:08.216294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:08.216969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:08.217419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:08.217844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:08.246953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-343ceb8a-97d8-4dfc-8087-618267c2efe7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043140 Sep 18 06:32:08.247494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 730/1427] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:08 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:09.177344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:09.177956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:09.184598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.185076 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.185511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.186191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.186290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:09.186709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.186892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:09.187078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.193304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.193783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.194223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.194938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.195393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.195816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.383206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:09.386184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:09.391707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ef89fddb-94b0-477f-9a87-ca92d81a0bc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.218248 Sep 18 06:32:09.392483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 698/1428] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:09 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:09.395557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:09.396996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:09.397375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:09.398161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b8c5becb-49ba-4fd3-afaa-b33a0c263957 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215869 Sep 18 06:32:09.409289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 731/1429] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:09 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:09.409476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:09.409592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:09.409711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.409805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:09.409896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.409896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.409896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.409896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.410205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.412604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.413098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.414885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.415601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.416070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.416506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.447493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-02264115-9e5a-448c-babe-cdb0ffd0187f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053878 Sep 18 06:32:09.447988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 699/1430] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:09.451685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d1b63fa0-bb12-4d0d-8e00-106ffdd2042b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051308 Sep 18 06:32:09.452778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 732/1431] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:09 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:09.457548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:09.459801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:09.465871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.466361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.466837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.467541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:09.467998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:09.468472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:09.499021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7cc4dea6-de95-450e-9794-35cc5bae0ccf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049995 Sep 18 06:32:09.500118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 700/1432] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:09 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:10.414734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:10.415337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:10.422421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.422919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.423345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.423867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:10.424283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.424452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.424559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:10.424868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.430800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.431296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.431714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.432389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.432838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.433259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.617956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:10.624918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:10.625965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e5a8041a-5586-416b-b291-9e86b02bb7d1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.215163 Sep 18 06:32:10.626543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 733/1433] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:10 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:10.633233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:10.633697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:10.635157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:10.635394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:10.636312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e1571363-d8a8-40ca-94af-39e767c3d78c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.216851 Sep 18 06:32:10.647995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 701/1434] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:10.648166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.648166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.648166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.648166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.648504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.648504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.651654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:10.653783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:10.660449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.660936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.661374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.662071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.662522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.662984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.674316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6c890ffb-2b3f-4a3b-9357-e8bb6dbba1ea tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044699 Sep 18 06:32:10.674847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 734/1435] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:10.679302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:10.681408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:10.686959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.687437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.687870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.688551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:10.689006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:10.689432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:10.695150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f9e23ded-7f6d-4c70-9581-4a9a5e59d579 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057001 Sep 18 06:32:10.695714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 702/1436] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:10 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:10.719385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9c79f6ae-1474-4871-b74e-b3156e7cd0b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043525 Sep 18 06:32:10.719919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 735/1437] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:10 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:11.650353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:11.650996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:11.658145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.658660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.659106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.659795 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.660244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.660666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.662063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:11.662706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:11.669072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.669548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.669985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.670701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.671151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.671573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.854262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:11.857656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:11.862162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d22caeb3-8c7c-4061-96cb-0948f64aeac9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216467 Sep 18 06:32:11.862766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 703/1438] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:11 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:11.866423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:11.867370 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:11.867905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:11.869065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c3227c5d-b24c-4991-842a-cfd0d7ade24e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.210835 Sep 18 06:32:11.869533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:11.869843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 736/1439] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:11 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:11.875116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:11.877187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:11.883250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.883733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.884160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.884858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.885307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.885725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.886027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.886510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.886971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.887665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.888120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.888541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.916564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a48c92ea-edad-472b-bac3-15657c934005 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045797 Sep 18 06:32:11.917122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 737/1440] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:11.917778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2178450f-4246-421a-badb-26609a46bdcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054003 Sep 18 06:32:11.918545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 704/1441] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:11.922011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:11.924115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:11.929632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.930104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.930533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.931250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:11.931698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:11.932116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:11.961319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-168f6ae0-23af-4430-8c41-e27a98a5bf4b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043170 Sep 18 06:32:11.961860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 738/1442] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:11 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:12.885289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:12.885930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:12.892810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:12.893337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:12.893780 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:12.894287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:12.894698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:12.894920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:12.895036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:12.895380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:12.901229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:12.901702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:12.902134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:12.902847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:12.903313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:12.903719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:12.906282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:13.088661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:13.095370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:13.097375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dc7eb95f-37c2-4ead-a86d-c39a0d38cce3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.215757 Sep 18 06:32:13.098256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 705/1443] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:12 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:13.103013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:13.104154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:13.105227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:13.105740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:13.107037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c5dfe568-e479-4518-9e1c-ab2409fff54a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.216516 Sep 18 06:32:13.107805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 739/1444] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:12 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:13.111908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:13.112390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:13.112847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:13.113543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:13.113994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:13.114415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:13.123725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:13.126554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:13.132546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:13.133022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:13.133455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:13.134139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:13.134598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:13.135034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:13.144911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2ee3322e-8ae1-4268-a492-d75ea5bed432 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045584 Sep 18 06:32:13.145470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 706/1445] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:13 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:13.150086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:13.152282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:13.157863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:13.158342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:13.158808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:13.159506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:13.159936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:13.160354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:13.164352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c9c80f2a-730f-4bc1-bfb5-9d53e64e2797 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044637 Sep 18 06:32:13.164913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 740/1446] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:13 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:13.189978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0ddea61a-27aa-4ee2-a00e-52e0b3bcaf25 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043529 Sep 18 06:32:13.190521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 707/1447] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:13 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:14.125862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:14.126473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:14.133063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.133550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.133988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.134708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.134850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:14.135245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.135495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:14.135604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.142392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.142904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.143346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.144038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.144492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.144919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.347007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:14.354707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:14.355207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9fc6a6da-1504-4e3e-8b29-2f908455fe42 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.233987 Sep 18 06:32:14.355786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 741/1448] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:14 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:14.360831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:14.363061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:14.368781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.369264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.369701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.370390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.370868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.371293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.376128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:14.377534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:14.378725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-71e6b77a-4b9c-42f3-8a06-56297924446a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.247744 Sep 18 06:32:14.379281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 708/1449] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:14 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:14.383915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:14.386039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:14.391900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.392383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.392820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.393501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.393948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.394374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.401908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c608796a-9de0-4a2c-aa1a-c519ce191673 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045094 Sep 18 06:32:14.402481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 742/1450] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:14.407024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:14.409182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:14.415160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.415854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.416350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.417055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:14.417616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:14.418051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:14.423766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-71999ace-0f17-422d-ba1b-97a4da0e47e9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043470 Sep 18 06:32:14.424330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 709/1451] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:14.448356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7147eab9-7140-4b06-9e2d-1a166166a82e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044848 Sep 18 06:32:14.448913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 743/1452] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:14 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:15.379457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:15.380070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:15.386425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.386950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.387384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.388061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.388516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.388936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.403639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:15.404355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:15.411335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.411815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.412250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.412942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.413390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.413807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.575564 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:15.583248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ff54a778-b459-439d-b69b-c2efbddd5950 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.207629 Sep 18 06:32:15.583816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 710/1453] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:15 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:15.588333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:15.590461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:15.596662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.597154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.597291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:15.597590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.598284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.598762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.599182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.606051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:15.607523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:15.609333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4ce0bb97-3ee5-4ddc-ac37-18805cafc326 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.208925 Sep 18 06:32:15.609888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 744/1454] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:15 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:15.614870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:15.616220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:15.617474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:15.624220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.624902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.625519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.626372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.627006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.627685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.630320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1c2ec1ba-5e8e-4d7e-a906-ede9ebd25093 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045462 Sep 18 06:32:15.630853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 711/1455] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:15.635358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:15.637496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:15.643100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.643586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.644016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.644698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:15.645144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:15.645562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:15.657196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4ffe0dfa-4417-40fb-a83e-c6fe4e645f90 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046107 Sep 18 06:32:15.657765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 745/1456] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:15 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:15.675454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fa76e5a2-bd1e-4620-95e4-2e0632102948 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043603 Sep 18 06:32:15.676016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 712/1457] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:15 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:16.607211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:16.607816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:16.614757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.615229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.615686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.616360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.616804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.617223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.635053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:16.635665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:16.642119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.642632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.643075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.643802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.644252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.644679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.820681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:16.824162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:16.829135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bfa30c48-9cdb-481f-aed2-851e49f30981 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.225834 Sep 18 06:32:16.830057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 746/1458] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:16 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:16.833115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:16.834338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:16.834863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:16.835553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c06e775d-b1cb-4074-b65d-4fc4e18b9890 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.204431 Sep 18 06:32:16.837870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 713/1459] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:16 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:16.838011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:16.841751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:16.843606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.844082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.844517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.844775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:16.845206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.845650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.846069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.851464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.851954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.852392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.853135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.853555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.853982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.883136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-33d1acd6-37b3-42c7-a65f-4df44888ef31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051984 Sep 18 06:32:16.883702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 747/1460] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:16.885194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9df0306b-8129-4c46-873e-4b511e967e63 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048059 Sep 18 06:32:16.889744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 714/1461] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:16.889886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:16.890689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:16.896351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.896823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.897251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.897928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:16.898397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:16.898838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:16.927624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-83c8eb3e-e605-446b-9f6c-c32a0ed6b3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042973 Sep 18 06:32:16.928168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 748/1462] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:16 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:17.852847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:17.854879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:17.859869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:17.860350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:17.860594 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:17.861035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:17.861414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:17.861738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:17.862195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:17.862638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:17.869225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:17.869444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:17.869859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:17.870561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:17.871040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:17.871522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:18.063216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:18.066365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:18.070991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-da4f9b0a-c375-4e5a-8c37-5b80bbf0d4c9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.222103 Sep 18 06:32:18.071668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 715/1463] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:17 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:18.075109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:18.076215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:18.076560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:18.077809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bad0e226-5063-47b7-9c26-8f12f7de31ee tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.221767 Sep 18 06:32:18.078519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 749/1464] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:17 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:18.082877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:18.084964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:18.090676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:18.091156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:18.091345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:18.091595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:18.092278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:18.092882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:18.093312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:18.097362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:18.097847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:18.098422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:18.099172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:18.099628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:18.100055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:18.122406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3189a08f-c947-45a1-8432-06938169d5cb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042894 Sep 18 06:32:18.123180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 750/1465] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:18 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:18.128276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:18.130399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:18.130540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4f8a547a-3b1a-457e-9748-8df3a8a6a060 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057728 Sep 18 06:32:18.131008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 716/1466] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:18.136006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:18.136487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:18.136923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:18.137606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:18.138063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:18.138487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:18.167077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6662a07c-d049-4905-963f-fb16ad8f7fd6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042873 Sep 18 06:32:18.167634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 751/1467] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:18 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:19.094520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:19.095147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:19.101377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.101838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.102266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.102973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.103079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:19.103485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.103689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:19.103826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.105538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:19.110718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.111204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.111646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.112339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.112791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.113216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.309087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:19.316615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:19.317292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-38884c85-ba6f-4c59-bc5f-e82793ef2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.227295 Sep 18 06:32:19.317903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 717/1468] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:19 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:19.322747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:19.325178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:19.325877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:19.327948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:19.329267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aa429c9a-c5bb-4c54-826d-a9efadcbe881 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.230325 Sep 18 06:32:19.339580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 752/1469] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:19 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:19.340019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.340019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.340019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.340019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.340511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:19.340597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.340597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.340775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:19.343153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.343648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.344088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.344854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.345253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.345678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.365833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-03a32622-b848-45cc-8dfe-7c427f2856f8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046772 Sep 18 06:32:19.366393 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 718/1470] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:19 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:19.370972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:19.373378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:19.376032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-100da4fd-1a1c-48f9-9b19-39b523afe857 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045175 Sep 18 06:32:19.376614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 753/1471] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:19.379604 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.380090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.380524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.381217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:19.381671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:19.382097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:19.412258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-965db2bb-9c9e-4db6-a003-daf0814d9531 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044794 Sep 18 06:32:19.413050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 719/1472] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:19 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:20.341259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:20.341891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:20.349472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.350032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.350473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.351190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.351645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.352092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.354856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:20.355472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:20.361826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.362314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.362787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.363480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.363932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.364359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.552428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:20.556589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:20.560278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-43a6bdc9-465f-44a8-82ff-f8765db9a7e6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.222755 Sep 18 06:32:20.560854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 754/1473] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:20 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:20.565488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:20.565730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:20.566952 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:20.567960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:20.568090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-667a55ee-3869-4fa7-80f8-08995e03a1fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.217128 Sep 18 06:32:20.568793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 720/1474] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:20 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:20.573176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:20.573974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.574461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.575044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.575331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:20.575737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.576315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.577330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.581567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.582002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.582431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.583191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.583624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.584044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.603199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:20.608848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ddb1858d-b0f7-4389-869b-ac4a6a7acf7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047024 Sep 18 06:32:20.609419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 755/1475] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:20 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:20.614108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:20.616444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:20.617964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c42d242f-fe3f-4364-a859-2a48c72372d4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048322 Sep 18 06:32:20.618525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 721/1476] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:20 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:20.622446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.623240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.623685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.624462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:20.625077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:20.625518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:20.656518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fc7779b0-da9b-4adb-9f40-54dd97d04e70 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046124 Sep 18 06:32:20.657074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 756/1477] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:20 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:21.593964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:21.594597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:21.601084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.601231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:21.601629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.601838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:21.601998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.602722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.603178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.603600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.608816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.609300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.609738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.610459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.610952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.611376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.800581 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:21.804863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:21.809507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:21.810968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:21.812139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-76d86513-721d-44ea-88c7-fdfd0e6b12ad tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.222781 Sep 18 06:32:21.813059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 722/1478] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:21 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:21.817182 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:21.817788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-12236c94-ce87-451a-83cb-3539fdbd4388 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.215479 Sep 18 06:32:21.818916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 757/1479] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:21 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:32:21.819371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:21.825279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.825759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.826183 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.827403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.827855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.828274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.833902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:21.836658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:21.843279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.843757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.844218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.845041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.845503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.845922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.859073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-770a8ff3-d6e9-4cdf-9ff6-a78364ba3900 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045334 Sep 18 06:32:21.860180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 723/1480] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:21 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:21.864132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:21.866257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:21.871904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.872385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.873076 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.873797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:21.874247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:21.874693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:21.876943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-44c16f26-7f7d-4501-bed0-41d359865bb8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056392 Sep 18 06:32:21.877508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 758/1481] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:21.903803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2dafb5d8-481f-44ce-a5f9-874bec171cb3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043173 Sep 18 06:32:21.904344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 724/1482] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:21 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:22.842682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:22.843301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:22.843677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:22.843804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:22.849607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:22.849755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:22.850083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:22.850204 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:22.850517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:22.850659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:22.851227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:22.851329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:22.851675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:22.851789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:22.852121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:22.852235 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:23.043532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:23.048609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:23.053303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:23.054831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:23.056027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e72a0119-a4d7-4606-b34b-df660da3f523 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.217309 Sep 18 06:32:23.056475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9f30b7b6-fe82-4168-a050-6dda1ea767cb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.217647 Sep 18 06:32:23.056587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 759/1483] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:22 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:23.062716 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 725/1484] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:22 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:23.062868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:23.063336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:23.067197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:23.069359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:23.069782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:23.069915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:23.070256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:23.070970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:23.071416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:23.071841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:23.076874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:23.077361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:23.077806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:23.078502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:23.079040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:23.079468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:23.100744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-122550ae-5d7b-48b1-a0c4-b9935bad4501 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043081 Sep 18 06:32:23.101224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 760/1485] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:23 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:23.105684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:23.107803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:23.111208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5e9912e4-427b-491f-a975-e9dfd5527c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053062 Sep 18 06:32:23.111909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 726/1486] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:23.114081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:23.114555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:23.115013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:23.115701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:23.116146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:23.116566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:23.145100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bdfc48ef-f174-4eb5-a21c-ab2494d642e2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042925 Sep 18 06:32:23.145638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 761/1487] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:23 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:24.079624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:24.080253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:24.081355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:24.081999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:24.086481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.086998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.087440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.088134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.088406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.088589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.088886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.089015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.089330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.090019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.090471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.090921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.283822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:24.287183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:24.291722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5646f03e-8be0-454c-894c-2af9b5b11769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.216714 Sep 18 06:32:24.292283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 727/1488] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:24 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:24.296096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:24.297547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:24.297853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:24.298759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8c0340bf-3445-410f-83b7-c8778d770389 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.221183 Sep 18 06:32:24.310798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 762/1489] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:24 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:24.310977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:24.311068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:24.311068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:24.311255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.311255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.311255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.311255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.311547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.311547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.311696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.313151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.313291 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:24.313385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.313385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.313649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.313961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.339073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b69de160-1181-4a69-87b8-147d755ad004 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044825 Sep 18 06:32:24.339559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 728/1490] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:24 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:24.344164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-92fe0b86-75cd-4254-a6ba-ada0b3b651d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043844 Sep 18 06:32:24.344731 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 763/1491] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:24 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:24.345294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:24.347637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:24.353429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.353917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.354355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.355070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:24.355522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:24.355946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:24.384476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-98b3a7f0-fedf-49cf-924d-278561567293 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043937 Sep 18 06:32:24.385021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 729/1492] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:24 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:25.315219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:25.315820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:25.322270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.322788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.323222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.323927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.324359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.324546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:25.324963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.325169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:25.332226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.332711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.333149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.333849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.334324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.334766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.525556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:25.529823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:25.533613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-af818ff9-2352-4608-8c94-d4727d861986 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.222723 Sep 18 06:32:25.534199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 764/1493] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:25 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:25.538993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:25.539405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:25.541056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:25.541197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:25.542241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f91e7117-1a48-49cc-9f40-7a91d812ffc4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.221586 Sep 18 06:32:25.550629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 730/1494] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:25 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:25.550797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.550797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.550982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:25.551069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.551069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.551069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.551069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.551356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:25.556406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.556901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.557517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.558223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.558709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.559146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.580377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d0b4079f-75bd-404e-9c44-473b03e659f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045032 Sep 18 06:32:25.580868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 765/1495] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:25 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:25.585384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:25.587503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:25.589830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-332ba3c5-b265-4805-8227-e6447d2e7ac2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045978 Sep 18 06:32:25.590397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 731/1496] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:25.593157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.593637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.594065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.594773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:25.595217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:25.595632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:25.625677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a61faf1a-6821-49bc-b7b2-79c23dac44db tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043557 Sep 18 06:32:25.625977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 766/1497] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:25 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:26.557581 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:26.558385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:26.564751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.565237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.565677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.566371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.566859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.567287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.568798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:26.569595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:26.577130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.577693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.578231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.579108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.579599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.580176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.775060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:26.779053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:26.783142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-064e50e3-b49d-4fc1-8c6c-3981fabb75a0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.229549 Sep 18 06:32:26.783721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 732/1498] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:26.788460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:26.788621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:26.790521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:26.790790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:26.791726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-94025919-c603-4854-9cf3-fca451a3ef26 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.227128 Sep 18 06:32:26.792427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 767/1499] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:26 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:26.796766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:26.798873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:26.799107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:26.804995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.805379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.805624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.806158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.806961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.807267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.807543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.807711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.808102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.808412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.808870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.809299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.838396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2ecb06c7-e151-46fe-820a-0e35f5cd639e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053634 Sep 18 06:32:26.838992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 733/1500] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:26 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:26.840347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e068ed72-216a-4f91-8308-a91b8711f258 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047056 Sep 18 06:32:26.841185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 768/1501] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:26.843546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:26.845670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:26.851344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.851807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.852246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.852929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:26.853382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:26.853805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:26.883208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d463784d-2f13-4a11-ae25-93c12ec60b20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043221 Sep 18 06:32:26.883766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 734/1502] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:26 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:27.806075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:27.806643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:27.813155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:27.813678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:27.814185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:27.814936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:27.815439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:27.815954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:27.816864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:27.817474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:27.823866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:27.824343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:27.824776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:27.825465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:27.825919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:27.826866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:28.020286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:28.024363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:28.028198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc39f6c0-1926-49e1-b434-3e7d4c60fba9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.226098 Sep 18 06:32:28.028778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 769/1503] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:27 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:28.033456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:28.033705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:28.035450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:28.036395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:28.036624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4adc49a7-4879-4b78-8c16-ad9fde6d0095 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.223667 Sep 18 06:32:28.037308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 735/1504] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:27 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:28.041722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:28.043834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:28.049771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:28.049930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:28.050267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:28.050399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:28.050722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:28.050852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:28.051420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:28.051531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:28.051866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:28.051971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:28.052291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:28.052399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:28.082029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-83226d54-169d-41bd-9c44-deb219154bb2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043735 Sep 18 06:32:28.082343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a34be8e7-2bf3-4496-afc2-168e6820cd87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052488 Sep 18 06:32:28.086300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 736/1505] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:28 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:28.086448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 770/1506] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:28 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:28.087181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:28.089335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:28.095550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:28.096030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:28.096466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:28.097148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:28.097598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:28.098048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:28.126776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7bf677ad-8c41-4947-98e9-9f14dfa43957 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043097 Sep 18 06:32:28.127324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 737/1507] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:28 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:29.053251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:29.053866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:29.061019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.061565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.062042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.062768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.063222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.063458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:29.063893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.064264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:29.071116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.071673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.072175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.072947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.073484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.073978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.301704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:29.305621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:29.310217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4a004646-882c-455a-b6ee-f1105fd71bb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.261038 Sep 18 06:32:29.310764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 771/1508] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:29 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:29.316252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:29.317120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:29.318599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:29.319668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:29.320562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-398fd5e1-1464-4d0c-85c9-c936b430f427 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.261549 Sep 18 06:32:29.321886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 738/1509] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:29 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:29.325649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.326140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.327258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.327940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:29.328088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.328402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.333825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:29.336679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.340970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.341454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.342488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.344157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.344157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.344157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.354833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:29.374957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e461dedd-4fca-4f6b-be97-ec883aae6539 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062083 Sep 18 06:32:29.375188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f92302ed-3a7a-4873-b2d8-7893130cd693 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051920 Sep 18 06:32:29.375571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 739/1510] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:29 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:29.380037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:29.382226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:29.383616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 772/1511] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:29.387964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.388447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.389385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.390070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:29.390514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:29.390961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:29.420672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1a4cf40d-edc0-4e7a-9e37-420d82fb64b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044086 Sep 18 06:32:29.421216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 740/1512] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:29 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:30.336206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:30.336737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:30.343994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.344796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.344952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.345584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.346034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.346715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.351234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:30.351894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:30.358420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.358941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.359384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.360086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.360599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.360976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.546520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:30.550543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:30.554799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-021d1139-0f45-4f64-84e3-dad130f8206a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.222518 Sep 18 06:32:30.555376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 773/1513] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:30 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:30.559260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:30.561202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:30.562367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d21c44ea-4963-4593-96a9-dfd436764596 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.215122 Sep 18 06:32:30.575070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 741/1514] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:30 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:30.575281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:30.575281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:30.575486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:30.575486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:30.575663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.575663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.575663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.575663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.576048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.576048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.579037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.579521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.579958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.580859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.581345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.581905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.606162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a37acadb-c252-4bd4-8af2-43278da4c471 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045640 Sep 18 06:32:30.606662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 774/1515] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:30 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:30.611157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:30.612690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-68a15048-945f-4eee-b8d5-67b4956ba809 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048719 Sep 18 06:32:30.613251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:30.617444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 742/1516] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:30 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:30.618875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.619348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.619775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.620448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:30.620894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:30.621310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:30.650860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9ce0ce69-6560-4410-a178-ca161775da83 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043178 Sep 18 06:32:30.651439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 775/1517] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:30 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:31.580202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:31.589749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:31.590459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.590983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.591417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.592108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.592230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:31.592654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.592849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:31.592994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.599439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.599920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.600385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.601077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.601623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.601954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.793006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:31.800784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:31.801989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-92037299-5ac7-4c89-b4bd-a3af6088e143 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.226004 Sep 18 06:32:31.802567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 743/1518] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:31 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:31.807721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:31.809853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:31.812247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:31.813389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b076d2b2-df2f-432c-8c90-c166c1887d94 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.225975 Sep 18 06:32:31.813585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:31.814128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 776/1519] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:31 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:31.819670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.820152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.820584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.821274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.821721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.822143 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.825025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:31.827525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:31.833441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.834002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.834708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.835600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.835916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.836343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.849736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:31.855884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f26934a3-4398-46ac-905f-60c5f0b0246c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052216 Sep 18 06:32:31.856457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 744/1520] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:31.862432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:31.864864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:31.867075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-35cde4df-edd3-4a1c-aff9-76a918daba74 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046017 Sep 18 06:32:31.867836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 777/1521] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:31 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:31.871694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.872229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.872935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.873772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:31.874303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:31.874821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:31.914379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4b6b14b8-4218-41c5-b5c2-94191fbe9d33 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056881 Sep 18 06:32:31.914957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 745/1522] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:31 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:32.826196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:32.827476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:32.833745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:32.834226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:32.834695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:32.835418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:32.835843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:32.836264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:32.839762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:32.840674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:32.848119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:32.848691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:32.849224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:32.850449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:32.851009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:32.851686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:33.144169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:33.152081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-35536a5a-1b14-4d6b-aaa7-e8b7826d7680 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.329711 Sep 18 06:32:33.152662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 778/1523] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:32 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:33.157186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:33.159486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:33.165629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:33.166153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:33.166614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:33.167315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:33.167769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:33.168196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:33.174913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:33.184300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:33.186465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:33.187835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8d080df6-4941-4cb8-97cf-1c7505de993d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.353181 Sep 18 06:32:33.188700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 746/1524] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:32 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:33.194193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:33.196711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:33.203076 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:33.203922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:33.204770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:33.205471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f50de95b-6cd3-43c1-88da-2d3540a29d35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050737 Sep 18 06:32:33.205596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:33.206079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:33.206390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 779/1525] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:33 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:33.206752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:33.215210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:33.217361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:33.223768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:33.224247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:33.224677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:33.225397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:33.225847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:33.226274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:33.239310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-883c77e8-7a54-4c59-8e3c-0b705073582b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049039 Sep 18 06:32:33.239892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 747/1526] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:33 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:33.259230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-64c795cc-d29b-4075-92e4-e33fcf6cdc7d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051746 Sep 18 06:32:33.259798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 780/1527] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:33 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:34.175999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:34.176627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:34.183106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.183592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.184028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.184711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.185166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.185587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.213486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:34.214138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:34.220730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.221394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.221832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.222520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.222993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.223415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.377239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:34.385118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ce6bacb9-e2ae-4d55-b67c-562adcc57df3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.213088 Sep 18 06:32:34.385694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 748/1528] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:34 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:34.391381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:34.393257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:34.399166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.399658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.400468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.400796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.401255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.401686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.410917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:34.419961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:34.421583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:34.422815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-76a2ffbf-966e-49cb-b7e7-c68874902952 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.213471 Sep 18 06:32:34.423376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 781/1529] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:34 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:34.428177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:34.430405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:34.431428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b3414d4e-2241-48aa-b056-918ec1971f8e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044705 Sep 18 06:32:34.433818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 749/1530] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:34.437442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:34.437904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.438390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.438848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.439608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:34.439728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.440117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.440540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.445285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.445772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.446210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.446958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:34.447427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:34.447864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:34.472098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4b368cf6-0ac1-417f-b2da-fa1375c25f17 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047607 Sep 18 06:32:34.472672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 782/1531] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:34 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:34.480130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-57a05f17-93b8-4127-8e36-abec14836f7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046826 Sep 18 06:32:34.480683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 750/1532] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:34 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:35.411749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:35.412354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:35.418954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.419433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.419866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.420548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.420999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.421415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.445167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:35.447464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:35.460876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.461398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.462104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.462990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.463486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.463952 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.497777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0623e88c-cff4-42b2-ba49-ef6eef07d546 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056407 Sep 18 06:32:35.498375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 751/1533] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:35.502828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:35.503485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:35.509750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.510328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.510879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.511656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.512180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.512732 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.621332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:35.629547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-742b8785-3c0a-4821-86d1-ff285062f45d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.222035 Sep 18 06:32:35.630215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 783/1534] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:35 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:35.637691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:35.640395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:35.649106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.649626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.650183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.650948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.651406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.651832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.687479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3dd78c24-8e46-40c9-a9c7-f3cd99befa45 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055994 Sep 18 06:32:35.687992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 784/1535] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:35 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:35.692704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:35.694918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:35.702220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.703040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.703814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.705271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:35.705729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:35.706154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:35.741566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f9fe78a6-bf79-4cf6-8a21-8b1befe721b9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052482 Sep 18 06:32:35.742164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 785/1536] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:35 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:35.939783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:35.950754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:35.953274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:35.954473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cee74a55-d0ef-4220-bfef-58043c28a5d7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.455131 Sep 18 06:32:35.955068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 752/1537] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:35 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:36.510708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:36.513040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:36.520708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.521228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.521670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.522410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.522924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.523369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.552455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-32ace772-14e3-4bf9-879c-525f59bfde35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046885 Sep 18 06:32:36.553021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 786/1538] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:36.655088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:36.655698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:36.662061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.662541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.663004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.663682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.664129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.664549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.699195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:36.701981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:36.708067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.708555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.708995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.709694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.710149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.710592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.740123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6809f363-5172-4ecd-b3b0-524bbc518d65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044729 Sep 18 06:32:36.740933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 787/1539] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:36 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:36.753865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:36.756100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:36.762007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.762489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.762973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.763695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.764694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.765121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.793572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:36.794074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ee0de8f4-005f-4048-a8ec-9edbb9d91112 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043989 Sep 18 06:32:36.794670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 788/1540] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:36 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:36.802854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c043b7af-86c3-44f9-9a37-ae0ca1789483 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151570 Sep 18 06:32:36.803415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 753/1541] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:36 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:36.979703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:36.980311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:36.986847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.987329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.987769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.988456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:36.988932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:36.989327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:36.994841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:37.118783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:37.128236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:37.129667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:37.130880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ef561710-e5ee-4c98-88cf-3164d33853a8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155028 Sep 18 06:32:37.131436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 789/1542] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:36 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:37.565141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:37.567357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:37.573337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:37.573821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:37.574258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:37.574970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:37.575424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:37.575847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:37.604954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d0cbad81-b1f3-48f7-9409-855a29f88517 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043982 Sep 18 06:32:37.605495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 754/1543] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:37 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:37.752491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:37.754781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:37.761176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:37.761652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:37.762082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:37.762789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:37.763239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:37.764319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:37.792797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e6d63677-b16b-454f-b703-d041ad10862a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044168 Sep 18 06:32:37.793357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 790/1544] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:37 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:37.806825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:37.808993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:37.817268 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:37.817745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:37.818177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:37.818890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:37.819336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:37.819754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:37.825973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:37.827275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:37.833085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:37.833567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:37.834000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:37.834711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:37.835157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:37.835572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:37.850815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e43d0e5a-62d2-43f7-b9ed-651b20266185 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047822 Sep 18 06:32:37.851360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 755/1545] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:37 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:37.963153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:37.970907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1fa7df3a-8441-4f5e-a6fe-0a3833d8c504 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148577 Sep 18 06:32:37.971417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 791/1546] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:37 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:32:38.155987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:38.156600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:38.162705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:38.163184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:38.163648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:38.164331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:38.164783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:38.165203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:38.292740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:38.301760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:38.303244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:38.304405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ad44937b-3b77-4411-88e1-8fa5bb73fae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152353 Sep 18 06:32:38.304956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 756/1547] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:38 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:38.617371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:38.619620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:38.626440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:38.626951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:38.627382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:38.628064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:38.628511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:38.628936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:38.658179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8b7f341a-304d-4473-b83f-46ebea6f0ec2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044802 Sep 18 06:32:38.658761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 792/1548] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:38 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:38.805116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:38.807272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:38.813243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:38.813725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:38.814163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:38.814883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:38.815340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:38.815766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:38.845351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f005a59f-3599-47d8-aadc-9ce39ed72ea4 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044160 Sep 18 06:32:38.845900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 757/1549] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:38 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:38.863758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:38.865895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:38.872508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:38.873021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:38.873456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:38.874135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:38.874599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:38.875044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:38.903421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6805dd8a-20a6-41fb-b22a-e6cc701747de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043629 Sep 18 06:32:38.903965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 793/1550] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:38 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:38.993601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:38.994214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:39.000520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.001000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.001437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.002119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.002596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.003024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.128535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:39.136385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5394ecaa-6efe-4202-ad02-c22716f3ef20 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146688 Sep 18 06:32:39.136943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 758/1551] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:38 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:39.330394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:39.330978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:39.336931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.337405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.337838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.338510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.338990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.339490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.468114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:39.476663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:39.478084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:39.479278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3ff38704-4cb3-4c4f-bfc9-54deb1557903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153820 Sep 18 06:32:39.479832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 794/1552] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:39 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:39.669868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:39.672089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:39.678009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.678487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.678958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.679644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.680099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.680526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.710241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9c7d481f-50b7-4028-9ea2-7fcbf9e1c85e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044351 Sep 18 06:32:39.710813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 759/1553] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:39 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:39.857983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:39.861613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:39.868238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.868875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.869150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.869841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.870296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.870775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.901554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1531aeca-4f3d-48e5-b730-f180f6455c21 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048123 Sep 18 06:32:39.902116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 795/1554] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:39 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:39.915869 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:39.918042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:39.925085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.925506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.925941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.926652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:39.927114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:39.927541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:39.956985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b6edf6a6-05c3-488f-ac30-8c628319710f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045043 Sep 18 06:32:39.957533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 760/1555] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:39 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:40.160174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:40.160783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:40.166776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.167255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.167694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.168376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.168827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.169257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.300597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:40.308286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d165e88c-a48b-429f-8f02-8bbd42978554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151974 Sep 18 06:32:40.308849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 796/1556] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:40 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:40.504181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:40.504735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:40.510767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.511270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.511690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.512379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.512833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.513260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.531248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:40.640097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:40.648582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:40.650016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:40.651219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1adf4275-f7bc-4d51-8fe4-7d01fd52ecb9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151035 Sep 18 06:32:40.651777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 761/1557] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:40 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:40.723043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:40.725214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:40.731342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.731822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.732259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.732942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.733394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.733816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.762788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8c5f429f-7fca-4095-ac8d-0e547354bcfb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043547 Sep 18 06:32:40.763272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 797/1558] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:40.913924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:40.916107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:40.922425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.922953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.923375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.924054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.924501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.924923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.955014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-96d3145b-2797-4b5f-afb9-05f40bdd0c4d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045047 Sep 18 06:32:40.955554 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 762/1559] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:40 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:40.969256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:40.971496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:40.977608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.978084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.978513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:40.979219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:40.979665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:40.980198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:41.009411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0e1b59cf-3fba-4386-9553-301e85b1d66d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044066 Sep 18 06:32:41.009952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 798/1560] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:40 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:41.331056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:41.331665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:41.337523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:41.338005 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:41.338433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:41.339141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:41.339590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:41.340009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:41.472422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:41.480262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-de259184-fd5a-413c-95ca-b0b644c40c9a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153067 Sep 18 06:32:41.480824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 763/1561] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:41 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:41.677057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:41.677663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:41.683822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:41.684366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:41.684862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:41.685618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:41.686129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:41.686618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:41.775384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:41.777658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:41.784449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:41.785018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:41.785528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:41.786296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:41.786927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:41.787431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:41.814157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:41.817883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-53b11b18-6451-4838-8c79-cd5ea7db4b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046497 Sep 18 06:32:41.818451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 764/1562] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:41.823102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:41.824534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:41.825693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-620d0c85-a996-4895-a595-076658ddc12b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152636 Sep 18 06:32:41.826237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 799/1563] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:41 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:41.967351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:41.969413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:41.975774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:41.976247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:41.976681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:41.977355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:41.977802 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:41.978220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.006758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-95a71e09-1bd1-424f-a64c-9ab70ef0117c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043434 Sep 18 06:32:42.007297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 765/1564] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:41 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:42.022624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:42.024823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:42.031258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:42.031742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:42.032179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.032867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:42.033320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:42.033746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.063615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b77867f9-bd5e-4cd8-a7c8-68997fb02283 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044958 Sep 18 06:32:42.064163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 800/1565] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:42 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:42.504543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:42.505154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:42.511189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:42.511666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:42.512099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.512779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:42.513227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:42.514328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.640694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:42.648355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1b9a4ebb-72de-4f04-94a7-9d8bb0327a4f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147686 Sep 18 06:32:42.648915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 766/1566] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:42 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:42.829393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:42.831528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:42.837447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:42.837925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:42.838364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.839086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:42.839544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:42.839972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.851001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:42.851642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:42.857991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:42.858467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:42.858924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.859613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:42.860062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:42.860484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:42.869875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4f6bd229-77a5-415c-86fd-9e81c80c4d1d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044484 Sep 18 06:32:42.870439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 801/1567] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:42.989411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:42.998804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:43.000286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:43.001456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9376f23e-c54d-4da8-b78d-8f40b760a06b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154324 Sep 18 06:32:43.002016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 767/1568] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:42 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:43.018739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:43.020987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:43.022950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:43.028589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:43.029071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:43.029508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:43.030222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:43.030706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:43.031133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:43.059856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5c1e32da-1279-44c4-ba27-eb17c95c5e82 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045121 Sep 18 06:32:43.061013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 802/1569] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:43 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:43.075896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:43.078054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:43.083994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:43.084419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:43.084854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:43.085539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:43.086000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:43.086424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:43.116438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6c42a520-a33b-4884-9d30-368d0151dc64 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044459 Sep 18 06:32:43.116982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 768/1570] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:43 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:43.672146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:43.672753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:43.680762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:43.681238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:43.681672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:43.682356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:43.682841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:43.683262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:43.811223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:43.818837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4bc5558c-7553-45ea-8eed-edde31cfd1a3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151045 Sep 18 06:32:43.819399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 803/1571] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:43 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:43.882694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:43.884900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:43.890926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:43.891417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:43.891859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:43.892552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:43.893010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:43.893443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:43.923743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c9a907ba-cfa4-49ef-ad2b-ca9e517d7b6a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045075 Sep 18 06:32:43.924297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 769/1572] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:44.026553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:44.027261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:44.034111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.034603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.035045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.035722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.036172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.036590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.072059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:44.074230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:44.080651 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.081142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.081581 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.082309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.082798 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.083227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.113518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f4c0c94c-0540-4f0c-bdf3-cf171fe5eca8 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045333 Sep 18 06:32:44.114087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 770/1573] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:44 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:44.128878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:44.131107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:44.136936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.137415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.137868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.138594 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.139046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.139466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.167306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:44.169391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d4fa1daf-88e2-465f-8289-1e19a13ddbc9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044347 Sep 18 06:32:44.170659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 771/1574] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:44 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:44.177990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:44.179482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:44.180721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f6ef1ac1-23ea-4e0b-84c5-6e8add15c8e9 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158044 Sep 18 06:32:44.181278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 804/1575] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:44 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:44.841959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:44.842595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:44.844544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:32:44.844672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:32:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:32:44.850848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.851332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.851769 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.852463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.852916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.853340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.936407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:44.939758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:44.945922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.946406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.946913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.947700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:44.948170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:44.948637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:44.974455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:44.979989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-385efbd1-a46f-4d46-8c27-e54065fcbdc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047684 Sep 18 06:32:44.980551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 805/1576] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:44.992258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:45.000285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-188c1417-c27b-4f25-aec6-66e21ce962d8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.162964 Sep 18 06:32:45.000844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 772/1577] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:44 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:45.125883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:45.127994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:45.134309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:45.134828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:45.135257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:45.135931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:45.136376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:45.136792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:45.166023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eb410cd4-e68b-46f2-a679-d76d3083c532 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044708 Sep 18 06:32:45.166563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 806/1578] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:45 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:45.182198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:45.184361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:45.190832 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:45.191312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:45.191743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:45.192429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:45.192878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:45.193299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:45.205628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:45.206240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:45.212008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:45.212487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:45.212920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:45.213603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:45.214056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:45.214476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:45.223412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-414fd2f8-bbcf-4cd0-abf2-b960a69be2f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045012 Sep 18 06:32:45.223982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 773/1579] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:45 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:45.353248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:45.362645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:45.364091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:45.365271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5fd85eec-6ff0-4807-a0f3-459717bcd2fe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.163507 Sep 18 06:32:45.365836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 807/1580] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:45 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:45.992684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:45.994862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:46.001077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.001555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.002019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.002745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.003315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.003796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.023142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:46.023758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:46.030337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.030857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.031296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.031984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.032435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.032861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.033856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5ddbbf2e-6aa5-4fef-90be-29ed86b637af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045133 Sep 18 06:32:46.034412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 774/1581] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:45 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:46.159898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:46.167675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-411ad9d8-449b-4ef8-bf9b-d2fbdd9cfabf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148373 Sep 18 06:32:46.168246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 808/1582] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:46 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:32:46.177245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:46.179519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:46.185922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.186476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.186836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.187534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.188223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.188653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.217589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d0c27509-3c5e-4768-bb19-3a04215abf23 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044289 Sep 18 06:32:46.218134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 775/1583] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:46 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:46.235907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:46.238177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:46.244152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.244574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.245013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.245733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.246190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.246640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.277040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fdc4739d-5f6b-4b21-b9d6-6cc000c7ef3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045197 Sep 18 06:32:46.277590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 809/1584] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:46 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:46.390289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:46.390926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:46.396994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.397479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.397918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.398621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:46.399079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:46.399502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:46.525306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:46.533815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:46.535301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:46.536461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cc5fdf23-6854-4478-bfb4-4e20dfd88c99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150707 Sep 18 06:32:46.537021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 776/1585] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:46 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:47.046596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:47.048790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:47.055467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.055946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.056379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.057057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.057506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.057926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.086704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3582a26c-1fe3-4dd9-b8aa-606d09f7ca75 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044223 Sep 18 06:32:47.087255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 810/1586] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:47.190525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:47.191163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:47.197550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.198044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.198481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.199380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.199839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.200264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.229317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:47.231474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:47.237798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.238271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.238736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.239423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.239869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.240285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.271102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c7dd20c3-dbc1-44fd-a466-f0b5f493089b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045543 Sep 18 06:32:47.271674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 811/1587] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:47 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:47.289791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:47.292152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:47.299459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.299856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.300287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.300999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.301431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.301848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.331819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34f4d906-3661-49eb-b1fb-2d1938311e43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046073 Sep 18 06:32:47.332572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 812/1588] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:47 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:47.339205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:47.347259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9cb83fb0-a0ca-4233-9587-e511ff06ca47 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.160536 Sep 18 06:32:47.347820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 777/1589] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:47 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:47.562385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:47.562967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:47.568806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.569283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.569721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.570398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:47.570875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:47.571293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:47.700356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:47.708900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:47.710344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:47.711519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4607f4b3-d7bf-4ec5-92bb-8de926b79bb7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153783 Sep 18 06:32:47.712080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 813/1590] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:47 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:48.099366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:48.101585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:48.107660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.108178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.108617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.109358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.109855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.110302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.113943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:48.140344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-71415e2b-3594-48c5-a19a-d773a2c25402 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044911 Sep 18 06:32:48.140833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 778/1591] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:48.284069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:48.286249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:48.292971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.293452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.293888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.294602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.295049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.295474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.323591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a4a06203-1a24-4f2b-bc74-1507cca5eb81 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043504 Sep 18 06:32:48.324135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 814/1592] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:48 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:48.344672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:48.346876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:48.353336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.353812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.354244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.354965 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.355418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.355841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.370495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:48.371165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:48.377264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.377743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.378178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.378966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.379426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.379852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.385488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ec303af2-92f4-47b9-a171-e52a8224fc43 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045353 Sep 18 06:32:48.386227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 779/1593] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:48 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:48.516132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:48.523824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a38639b5-23b7-4769-bb83-faa7a7a7845f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.157427 Sep 18 06:32:48.524384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 815/1594] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:48 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:32:48.737200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:48.737737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:48.743791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.744274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.744710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.745399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:48.745851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:48.746278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:48.872843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:48.881921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:48.883487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:48.884714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e0eb61e7-f2c1-40c9-8c29-ed9dcc40c7ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151488 Sep 18 06:32:48.885310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 780/1595] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:48 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:49.153211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:49.155956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:49.161954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.162431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.162895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.163580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.164036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.164461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.193686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c93f2580-f585-4716-a3f6-6b946ab968cd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044740 Sep 18 06:32:49.194226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 816/1596] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:49.337341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:49.339909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:49.346452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.347075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.347468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.348161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.348615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.349043 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.379983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8f90226e-7057-4e6a-804f-c0c20ee03281 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047782 Sep 18 06:32:49.380532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 781/1597] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:49 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:49.398426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:49.400634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:49.407456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.408099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.408559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.409896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.410003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.410348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.441508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9a2383a9-1cf9-40b8-97da-e7efbd4cc821 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047028 Sep 18 06:32:49.442245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 817/1598] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:49 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:49.547294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:49.547906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:49.553776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.554255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.554720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.555403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.555853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.556274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.685816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:49.694094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-11241330-be05-46fe-945b-d19f9a3f1ea7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151250 Sep 18 06:32:49.694681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 782/1599] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:49 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:49.909947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:49.910547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:49.916596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.917074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.917515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:49.918199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:49.918677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:49.919106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.058635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:50.067839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:50.069298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:50.070462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0bbd319f-00f1-4a56-b4bb-013be16890e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.164444 Sep 18 06:32:50.071075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 818/1600] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:49 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:50.206415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:50.208641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:50.210943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:50.216117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:50.216605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:50.217031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.217713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:50.218162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:50.218603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.246940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c9985298-29e9-42f9-8d98-53ebae81a108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044384 Sep 18 06:32:50.248244 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 783/1601] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:50.392502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:50.394567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:50.401006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:50.401565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:50.401912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.402612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:50.403072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:50.403497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.432329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f486025d-e035-46a8-9c26-e8cfead51177 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043895 Sep 18 06:32:50.432882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 819/1602] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:50 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:50.453305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:50.455612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:50.461746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:50.462238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:50.462705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.463399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:50.463861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:50.464288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.493689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dc8b146d-2bcc-454a-885b-67148a7f94bd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044435 Sep 18 06:32:50.494238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 784/1603] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:50 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:50.717667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:50.718277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:50.724273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:50.724760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:50.725197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.725881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:50.726332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:50.726791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:50.853245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:50.861754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-508e25ba-11d4-4c89-84c8-c40b2726adb1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148466 Sep 18 06:32:50.862319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 820/1604] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:50 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:51.097530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:51.098054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:51.103978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.104455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.104885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.105559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.106006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.106431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.234667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:51.243071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:51.244501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:51.245647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a6495ba3-4c82-443a-86c0-e2c409c5ec27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152136 Sep 18 06:32:51.246192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 785/1605] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:51 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:51.259941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:51.262411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:51.268956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.269443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.269891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.270599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.271074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.271501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.311794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dc359de6-e8a4-4087-899b-b7a639886c2f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055501 Sep 18 06:32:51.312661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 821/1606] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:51.445407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:51.447610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:51.454173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.454815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.455260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.456017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.456468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.456979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.486415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e662903e-fbae-4a52-8eb7-d6087dc37d12 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045059 Sep 18 06:32:51.486995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 786/1607] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:51 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:51.506860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:51.508999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:51.515986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.516465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.516899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.517574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.518017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.518437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.531740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:51.548137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9786bf2f-39e9-4638-8a53-9cdafb80864f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045264 Sep 18 06:32:51.548901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 822/1608] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:51 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:51.885201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:51.885865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:51.892610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.893110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.893530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:51.894220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:51.894690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:51.895114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.022302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:52.031242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c37bdb2b-d685-4483-979b-3d3d4f765309 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150071 Sep 18 06:32:52.031819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 787/1609] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:51 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:52.272662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:52.273320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:52.279838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:52.280325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:52.280772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.281457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:52.281913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:52.282340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.327156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:52.329326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:52.335674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:52.336163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:52.336606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.337300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:52.337758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:52.338186 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.369712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b9bc8ecd-c1e0-42eb-9ff8-aa9af56ba384 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046551 Sep 18 06:32:52.370285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 788/1610] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:52 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:52.413424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:52.422690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:52.424180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:52.425389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-911ad520-b8d9-460f-bf7b-cce75bc3622d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156732 Sep 18 06:32:52.425970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 823/1611] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:52 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:52.499237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:52.501371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:52.507348 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:52.507817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:52.508246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.509050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:52.509525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:52.509946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.538866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d1fa7036-2ad4-4bea-9338-286f69a0fc08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043752 Sep 18 06:32:52.539339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 789/1612] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:52 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:52.560856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:52.563053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:52.569531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:52.570011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:52.570451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.571177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:52.571633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:52.572063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:52.602391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6ad2d7e0-c951-427d-8823-a9503fc002f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045489 Sep 18 06:32:52.603055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 824/1613] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:52 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:53.054113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:53.054751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:53.061429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.061910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.062340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.063062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.063515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.063935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.191847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:53.199733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-87cbeb87-ce2d-4315-aded-6da4cddcdcc5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149456 Sep 18 06:32:53.200913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 790/1614] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:53 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:53.382400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:53.384541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:53.391012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.391494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.391931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.392621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.393076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.393504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.422656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c573738c-3c9f-4bcf-9d0d-d7ddeda6a367 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044240 Sep 18 06:32:53.423206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 825/1615] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:53.449039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:53.449650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:53.456600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.457081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.457511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.458199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.458670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.459093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.551419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:53.553665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:53.559703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.560264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.560710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.561404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.561945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.562269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.585972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:53.593243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-592d5d0d-98c5-4cf5-b288-74c0f5c93084 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045761 Sep 18 06:32:53.593822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 826/1616] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:53 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:53.596526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:53.597961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:53.599228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9650b7c4-fa41-4018-8340-2f9a6c56944a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154063 Sep 18 06:32:53.599769 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 791/1617] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:53 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:53.614369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:53.616515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:53.622808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.623282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.623716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.624393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:53.624987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:53.625466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:53.655956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7e6557b9-e421-4eb6-a898-958a3be01b62 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045237 Sep 18 06:32:53.656501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 827/1618] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:53 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:54.223836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:54.224448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:54.230394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.230901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.231333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.232509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.232958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.233384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.364248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:54.372877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-22be5ac8-3f32-4083-a3da-03cb64907bcd tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152035 Sep 18 06:32:54.373445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 792/1619] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:54 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:54.435001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:54.437151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:54.443097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.443577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.444014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.444694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.445144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.445565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.475607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-efbd75d4-4745-4fcf-a7d1-d5a60ae4c911 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044500 Sep 18 06:32:54.476130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 828/1620] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:54.604819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:54.607606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:54.613835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.614311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.614779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.615457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.615905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.616325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.624458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:54.625062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:54.631417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.631899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.632332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.633016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.633464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.633885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.645377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b4b9acc6-de5b-4e3b-bfa0-e1ccd38c7c57 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044490 Sep 18 06:32:54.645931 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 793/1621] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:54 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:54.668004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:54.670753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:54.676746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.677279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.677779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.678458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:54.679054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:54.679528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:54.704770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:54.710681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-170ebc12-d31c-456c-bddb-ee8493d2b5ee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046379 Sep 18 06:32:54.711388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 794/1622] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:54 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:54.761775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:54.770277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:54.771739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:54.772892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5c54fe11-7e4d-4163-95c0-9c3f0efa212c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152742 Sep 18 06:32:54.773446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 829/1623] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:54 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:55.395529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:55.396137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:55.402277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.402789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.403223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.403902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.404351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.404770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.487520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:55.489943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:55.496046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.496527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.497715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.498412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.498894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.499315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.529796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e17d1780-b346-4270-979b-acdbf6f8192e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046269 Sep 18 06:32:55.530375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 830/1624] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:55.538308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:55.546623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8b66bf5b-0ebb-46b2-a8ed-2214214d80ee tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155016 Sep 18 06:32:55.547249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 795/1625] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:55 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:55.657243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:55.659503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:55.666463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.667368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.667652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.668344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.669281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.669714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.698905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-29a20095-e944-419e-be98-38060319119c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045658 Sep 18 06:32:55.699464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 831/1626] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:55 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:55.722401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:55.724712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:55.731970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.732447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.732879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.733560 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.734009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.734431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.764090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1ad4dadf-c03e-4aa7-b16c-e03932d39793 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045592 Sep 18 06:32:55.764637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 796/1627] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:55 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:55.799459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:55.800069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:55.805960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.806448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.806918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.807602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:55.808053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:55.808477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:55.934835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:55.943397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:55.944843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:55.946003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-73dec95a-8847-4e90-87d2-d26bdfc0f767 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150469 Sep 18 06:32:55.946561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 832/1628] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:55 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:56.541803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:56.543981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:56.549933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.550419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.550887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.551572 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.552026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.552448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.568993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:56.569602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:56.576070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.576543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.576970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.577660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.578109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.578521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.582543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7b06d90f-2748-4edd-9df4-ca0a57669731 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044676 Sep 18 06:32:56.583133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 797/1629] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:56 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:56.706128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:56.711368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:56.713561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:56.713987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d0ca1b8d-e1be-47a9-b4a6-1f3493a3a68f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148777 Sep 18 06:32:56.714551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 833/1630] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:56 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:32:56.719562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.719984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.720421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.721115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.721566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.721989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.751488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-082c88f2-2a44-476b-9b79-8d02d7c78a79 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044012 Sep 18 06:32:56.752109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 798/1631] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:56 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:56.775231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:56.777398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:56.783262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.783736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.784415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.785098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.785548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.785968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.816134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7cf9fa0e-07ef-4ca1-bb41-62ecd7895ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044791 Sep 18 06:32:56.816678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 834/1632] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:56 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:56.970868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:56.971429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:56.977340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.977824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.978263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:56.978975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:56.979431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:56.979855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.108563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:57.116538 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:57.117977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:57.119189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-24c9c255-7793-46f6-9343-ebfc7f391f51 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152320 Sep 18 06:32:57.119744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 799/1633] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:56 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:32:57.594822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:57.596998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:57.602945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:57.603424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:57.603855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.604533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:57.604980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:57.605400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.634707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0bd97ad4-b24e-4436-abf3-95273fdcc7f4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044354 Sep 18 06:32:57.635248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 835/1634] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:57.737663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:57.738207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:57.744683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:57.745175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:57.745614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.746333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:57.747079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:57.747507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.763882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:57.766003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:57.772302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:57.772718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:57.773148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.773833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:57.774277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:57.774729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.805418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b0b75aef-38c4-4c24-a01c-512f11f9b08f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045572 Sep 18 06:32:57.805991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 836/1635] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:57 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:57.828752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:57.830943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:57.836850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:57.837333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:57.837763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.838449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:57.838928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:57.839349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:57.868880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-64eb5161-984c-4925-9e77-ad7f1115d6d2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044416 Sep 18 06:32:57.869636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 837/1636] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:57 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:32:57.877998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:57.885790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e3549e5f-fcfd-445e-a1f6-5176bdd93f89 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152068 Sep 18 06:32:57.886350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 800/1637] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:57 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:32:58.146811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:58.147429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:58.154011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.154496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.154961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.155650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.156103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.156527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.283840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:58.292449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:58.293930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:58.295158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0bc89a7b-a522-43df-b61e-3c8bc9217d99 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152301 Sep 18 06:32:58.295714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 838/1638] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:58 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:58.647214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:58.649412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:58.655424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.655903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.656379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.657065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.657522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.657951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.687709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-06ab1f4f-76f0-4721-92bf-d0736d15e412 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044458 Sep 18 06:32:58.688258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 801/1639] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:58 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:58.817520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:58.819654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:58.826093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.826570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.827067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.827732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.828186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.828617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.832278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:58.857994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3831a5af-128f-4354-9353-d19068d17429 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044477 Sep 18 06:32:58.858550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 839/1640] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:58 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:58.883175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:58.885384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:58.891706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.892189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.892624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.893310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.893763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.894216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.908720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:58.909338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:58.915197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.915680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.916113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.916801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:58.917252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:58.917671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:58.925982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d74fe0bc-c719-4e21-b486-8a19bba974b7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046672 Sep 18 06:32:58.926539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 802/1641] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:58 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:59.043532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:59.051815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2733cbce-045e-4b61-842a-0821d3c4884d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146889 Sep 18 06:32:59.052384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 840/1642] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:32:58 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:32:59.320527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:59.321136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:59.327675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:59.328161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:59.328591 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:59.329276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:59.329725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:59.330151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:59.462685 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:59.468990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:59.470433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:32:59.471624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8d2450cd-a52f-49a7-b7a4-99e2eb046903 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154988 Sep 18 06:32:59.472178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 803/1643] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:32:59 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:32:59.701872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:59.705115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:59.711516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:59.711992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:59.712432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:59.713117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:59.713564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:59.713984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:59.743271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8eb94b64-296d-4516-991f-f228fab93c88 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047277 Sep 18 06:32:59.743826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 841/1644] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:59 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:59.870342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:59.873099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:59.879127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:59.879607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:59.880054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:59.880740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:59.881190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:59.881614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:59.910957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6da6d93c-3587-49e9-855d-0d6fb88f3108 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044485 Sep 18 06:32:59.911614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 804/1645] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:59 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:32:59.938084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:32:59.940244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:32:59.946539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:59.947046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:59.947484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:59.948161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:32:59.948608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:32:59.949028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:32:59.978339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-866739e5-73e8-43a4-8f73-43a53d19dcab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045002 Sep 18 06:32:59.978847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 842/1646] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:32:59 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:00.074133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:00.074767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:00.080622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:00.081101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:00.081534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:00.082214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:00.082689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:00.083110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:00.210848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:00.218413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-221ae807-4934-46a5-a2ba-0fe61d878a91 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148125 Sep 18 06:33:00.219003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 805/1647] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:00 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:00.497228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:00.497840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:00.503857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:00.504340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:00.504778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:00.505487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:00.505938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:00.506360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:00.629872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:00.638430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:00.639898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:00.641071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-12f7ec2a-3029-4c20-88bd-8eab9df3dad1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.148094 Sep 18 06:33:00.641633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 843/1648] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:00 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:00.756469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:00.758603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:00.764543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:00.765022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:00.765458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:00.766134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:00.766604 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:00.767024 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:00.797380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-685013bf-e9c1-4db8-8d51-9a8660ed4e4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045011 Sep 18 06:33:00.797938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 806/1649] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:00.922961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:00.925117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:00.931110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:00.931591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:00.932027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:00.932711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:00.933163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:00.933593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:00.962600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5e86ae60-2e74-46e6-b5c0-c79c7e3f411c tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043573 Sep 18 06:33:00.963197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 844/1650] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:00 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:00.990855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:00.993015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:00.999828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.000313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.000749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.001436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.001886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.002313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.031877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e5fa109c-9f19-4208-8198-8896c35d7dd4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044953 Sep 18 06:33:01.032359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 807/1651] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:00 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:01.241666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:01.242279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:01.248703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.249185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.249619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.250301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.250778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.251204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.381666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:01.389958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-96895bc2-be03-4ed6-ad2d-4e656aa240d5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152119 Sep 18 06:33:01.390528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 845/1652] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:01 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:01.667530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:01.668139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:01.674502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.675034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.675452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.676134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.676595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.676999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.802644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:01.810174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:01.811978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:01.812468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:01.813421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:01.814615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-89ac6189-7887-467e-9dbd-5417260c6624 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150992 Sep 18 06:33:01.818549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.819048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.819478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.820166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.820613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.821036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.824705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 808/1653] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:01 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:01.850174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1112f174-1074-4b93-8cf7-12092809fa24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044741 Sep 18 06:33:01.850790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 846/1654] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:01.975577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:01.977668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:01.984343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.984820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.985250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:01.985927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:01.986376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:01.987056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.016102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b31a3b43-b063-4f34-82fd-fb1c5bd14645 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044466 Sep 18 06:33:02.016708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 809/1655] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:01 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:02.044681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:02.046908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:02.053337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:02.053819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:02.054255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.054982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:02.055435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:02.055863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.084181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4f6bdfb0-eb40-4695-9a0a-7e7158bab7fb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044093 Sep 18 06:33:02.084722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 847/1656] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:02 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:02.413010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:02.413632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:02.419582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:02.420068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:02.420505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.421189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:02.421668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:02.422093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.551666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:02.559480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4792d21c-8684-4fa4-b8a3-f026db2c0668 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150335 Sep 18 06:33:02.560044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 810/1657] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:02 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:02.839443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:02.839950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:02.846177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:02.846682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:02.847115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.847795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:02.848243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:02.848665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.862965 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:02.865142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:02.870967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:02.871957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:02.872403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.873090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:02.873542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:02.873967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:02.891158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:02.905088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e22079f9-5e26-4d79-8235-a2d9e57f1bc8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046066 Sep 18 06:33:02.905655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 811/1658] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:02.976395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:02.984996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:02.986456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:02.987637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2a171af6-fa3f-4578-8395-c06afdf0fddc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152270 Sep 18 06:33:02.988191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 848/1659] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:02 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:03.029225 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:03.031516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:03.038755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:03.039245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:03.039682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:03.040404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:03.040839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:03.041270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:03.071282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5da51abc-7887-44ff-be1a-775d88147caa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046585 Sep 18 06:33:03.071831 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 812/1660] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:03 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:03.095516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:03.097595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:03.103943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:03.104420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:03.104856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:03.105532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:03.106472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:03.106917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:03.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-126e5ac6-7ecc-466b-83bb-c956a8032c75 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044373 Sep 18 06:33:03.136559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 849/1661] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:03 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:03.583157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:03.583773 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:03.589737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:03.590991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:03.591193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:03.591881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:03.592400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:03.592821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:03.719833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:03.727530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-30912130-f57b-46c9-a720-3a266690bdc0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148185 Sep 18 06:33:03.728095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 813/1662] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:03 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:03.917479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:03.919653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:03.925657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:03.926131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:03.926561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:03.927265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:03.927710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:03.928131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:03.957848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2437b199-1648-4d02-8db8-9935f0a2b46d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044442 Sep 18 06:33:03.958398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 850/1663] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:04.013400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:04.013948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:04.019929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.020411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.020852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.021539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.021992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.022416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.085889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:04.088991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:04.095686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.096394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.096610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.097304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.097758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.098188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.127800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-18014c69-1a49-48e7-8c1f-2641379b6e97 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047930 Sep 18 06:33:04.132769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 851/1664] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:04 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:04.148125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:04.150305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:04.152168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:04.154483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:04.158771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.159264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.159722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.160402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.160877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.161305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.163173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:04.164617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:04.165784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3e7ba1d9-5771-493f-8056-b0a2e5f11bc0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156858 Sep 18 06:33:04.166341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 814/1665] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:04 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:04.191475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bdf0d6ac-7a22-4e0e-b7a9-1b22f3423697 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047154 Sep 18 06:33:04.192027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 852/1666] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:04 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:04.750831 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:04.751456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:04.757441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.757922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.758356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.759075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.759530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.760022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.885264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:04.892940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7c0fdc41-3314-4c05-a031-dcdfe774ce59 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146617 Sep 18 06:33:04.893504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 815/1667] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:04 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:04.970111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:04.972301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:04.978224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.978746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.979186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:04.979874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:04.980325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:04.981342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:05.010049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3bb2ef6c-a854-47b2-ac88-579cf0f6604d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043850 Sep 18 06:33:05.010530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 853/1668] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:04 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:05.140782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:05.143031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:05.149495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:05.149978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:05.150412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:05.151136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:05.151588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:05.152014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:05.181514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9043ae70-3f8b-4489-82ec-e213d1fc1171 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044808 Sep 18 06:33:05.182106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 816/1669] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:05 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:05.190893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:05.191505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:05.197809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:05.198312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:05.198870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:05.199562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:05.200017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:05.200440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:05.203548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:05.205733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:05.211453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:05.211936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:05.212367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:05.213050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:05.213493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:05.213912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:05.244132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-85fb4b85-fc1e-43d3-b13e-e6fd4025e0c1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044288 Sep 18 06:33:05.244696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 817/1670] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:05 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:05.308709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:33:05.308897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:33:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 (ip 10.222.0.20) !!! Sep 18 06:33:05.333232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:05.342598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:05.344062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:05.345227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d9a1125d-d48d-4d7a-90de-7e00639ccc18 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158155 Sep 18 06:33:05.345787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 854/1671] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:05 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:05.916586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:05.917112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:05.919132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:33:05.919275 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:33:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:33:05.925382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:05.925858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:05.926285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:05.926987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:05.927432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:05.927848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.025245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:06.028311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:06.034419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:06.034940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:06.035371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.036071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:06.036528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:06.036958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.065326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:06.068056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:06.070171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-feece043-b156-474d-8c05-6f765bbde9ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051241 Sep 18 06:33:06.070772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 855/1672] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:06.076117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6f30b40f-81a8-4ef0-b1be-e7b4c68ad31a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.163462 Sep 18 06:33:06.076673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 818/1673] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:05 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:06.193864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:06.195948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:06.202742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:06.203225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:06.204493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.204493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:06.204760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:06.205171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.234359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ae52edae-7464-46ef-8190-f91f7bdb0a92 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044492 Sep 18 06:33:06.234928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 856/1674] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:06 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:06.256790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:06.258983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:06.265512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:06.265999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:06.266433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.267158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:06.267614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:06.268041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.296726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0f5a443e-6ce4-43c1-a776-b85ce60ec03c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043996 Sep 18 06:33:06.297265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 819/1675] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:06 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:06.379714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:06.380320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:06.386289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:06.386814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:06.387257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.387937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:06.388390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:06.388816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:06.518980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:06.528027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:06.529474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:06.530667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8bd2b17c-07bc-4f54-afa7-a20f5611dc87 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154945 Sep 18 06:33:06.531230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 857/1676] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:06 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:07.083030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:07.085160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:07.091638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.092114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.092580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.093260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.093709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.094135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.098935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:07.099549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:07.105737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.106220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.106680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.107373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.107950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.108382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.123758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b7d942f1-ff18-40d2-a2b5-6205f772b644 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044676 Sep 18 06:33:07.124415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 820/1677] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:07.235094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:07.242947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5e08036e-7d7a-4b11-a2af-ae738f50f06a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147916 Sep 18 06:33:07.243510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 858/1678] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:07 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:07.245641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:07.248590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:07.254569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.255071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.255504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.256207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.256658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.257081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.287460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-afbde99b-f942-4a04-80bb-61c504a91146 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045223 Sep 18 06:33:07.288455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 821/1679] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:07 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:07.310382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:07.312575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:07.318499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.319009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.319443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.320126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.320578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.321000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.350194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-94e8ec8b-74e5-4b43-aaaa-1e97dc3c6a10 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045096 Sep 18 06:33:07.350767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 859/1680] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:07 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:07.556729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:07.557344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:07.563858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.564338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.564772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.565456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:07.565908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:07.566330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:07.692192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:07.700891 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:07.702322 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:07.703513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9254f3e6-f9cd-44c2-85a4-47dc76120fdc tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150717 Sep 18 06:33:07.704062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 822/1681] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:07 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:08.136181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:08.138386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:08.144397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.144871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.145332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.146008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.146454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.146903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.176079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-99782b26-7937-4911-98a3-63f6d940b9ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043876 Sep 18 06:33:08.176613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 860/1682] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:08.266890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:08.267506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:08.273900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.274384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.274856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.275543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.275997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.276424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.305211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:08.307430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:08.313767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.314243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.314704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.315386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.315837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.316251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.346024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d37a696a-cba7-4031-aa93-e3fc31455c8d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044681 Sep 18 06:33:08.346588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 861/1683] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:08 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:08.362154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:08.364353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:08.370145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.370668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.371146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.372090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.372541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.372958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.403525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3ebf901d-66a2-4754-8c39-bed43c32dda7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044303 Sep 18 06:33:08.404091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 862/1684] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:08 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:08.412484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:08.420344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e0f1bf4f-b591-4e50-b5de-531326d1b615 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.157331 Sep 18 06:33:08.420900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 823/1685] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:08 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:08.728394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:08.728932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:08.734853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.735328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.735758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.736438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:08.736881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:08.737300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:08.863182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:08.872626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:08.874050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:08.875241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2eefa9ee-3545-4d46-8ecd-069c5782469d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150872 Sep 18 06:33:08.875796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 863/1686] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:09.189228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:09.191434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:09.197510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.197988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.198419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.199132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.199582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.200004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.228387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-23fa6206-dc74-4131-8750-1f9e3e96dd26 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043106 Sep 18 06:33:09.228916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 824/1687] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:09.358741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:09.360901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:09.367365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.367847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.368284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.368967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.369420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.369844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.398821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f827a5b8-bfa2-4edd-881a-a55b3e296619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044632 Sep 18 06:33:09.399376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 864/1688] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:09 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:09.415702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:09.417854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:09.424844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.425325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.425759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.426442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.426925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.427352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.443278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:09.443822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:09.449711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.450198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.450678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.451354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.451814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.452240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.456503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8beb6696-8c0a-43ba-a189-b9dc0701e64d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044665 Sep 18 06:33:09.457077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 825/1689] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:09 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:09.584871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:09.593045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-320b2fc4-8d85-4194-9606-cc423088630a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153620 Sep 18 06:33:09.593609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 865/1690] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:09 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:09.900902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:09.901507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:09.908080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.908567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.909001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:09.909683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:09.910136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:09.910559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.035833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:10.044310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:10.045749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:10.046940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bb9b3074-13d9-4fdd-814b-0c21b1352a6c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149969 Sep 18 06:33:10.047497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 826/1691] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:09 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:10.241260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:10.243411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:10.250082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:10.250559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:10.251026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.251703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:10.252150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:10.252568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.282786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c8be77c0-4a1e-4069-a0ec-733974913ecd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045541 Sep 18 06:33:10.283334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 866/1692] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:10.410859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:10.413008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:10.418976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:10.419545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:10.419883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.420567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:10.421019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:10.421444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.451210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b233311a-12ce-4075-b893-98fc92a7f3d5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044356 Sep 18 06:33:10.451763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 827/1693] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:10 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:10.469659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:10.471809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:10.478105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:10.478596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:10.479034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.479710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:10.480152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:10.481232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.509849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-00ced774-b6fc-4254-8399-95dba43bcb45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045023 Sep 18 06:33:10.510328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 867/1694] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:10 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:10.615957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:10.616576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:10.622821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:10.623312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:10.623752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.624488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:10.625319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:10.626103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:10.630684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:10.755660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:10.763987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c01a08ed-cf60-4855-ba63-2d6e46d6783a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151827 Sep 18 06:33:10.764494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 828/1695] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:10 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:11.071828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:11.072355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:11.079044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.079523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.079961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.080644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.081094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.081546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.208246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:11.216676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:11.218117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:11.219320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e0abebed-e0dd-4866-a1ad-7ff224982d88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151550 Sep 18 06:33:11.219878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 868/1696] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:11 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:11.295321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:11.297480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:11.303497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.303975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.304405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.305087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.305537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.305985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.338338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d5c1c781-74f6-440c-b588-a1d62ece5ad7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047459 Sep 18 06:33:11.338841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 829/1697] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:11.463648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:11.465835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:11.471967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.472445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.472867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.473554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.474009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.474433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.504572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-725c6490-b565-487d-93b6-c787e1c1dd68 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044876 Sep 18 06:33:11.505127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 869/1698] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:11.521376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:11.523588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:11.529910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.530390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.530864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.531550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.532002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.532426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.562179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8f69ddc4-6d67-4d3b-986c-1b06061e5d38 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044861 Sep 18 06:33:11.562758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 830/1699] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:11 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:11.787549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:11.788153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:11.794677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.795163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.795597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.796284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:11.796735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:11.797158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:11.924040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:11.931813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-243f4735-65cb-4791-a868-cc61105fea83 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148174 Sep 18 06:33:11.932376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 870/1700] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:11 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:12.244438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:12.244972 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:12.251166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.251638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.252069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.252745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.253194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.253611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.350445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:12.352651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:12.358619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.359114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.359557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.360263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.360720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.361150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.384124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:12.391600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4a567019-a8e5-4ecd-b64e-a7c668cbd06e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045166 Sep 18 06:33:12.392174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 871/1701] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:12.393324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:12.394829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:12.395987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-23c1905d-7782-4e16-849f-49c768270286 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155535 Sep 18 06:33:12.396531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 831/1702] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:12 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:12.517632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:12.519775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:12.525912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.526389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.526853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.527534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.527978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.528395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.541650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:12.557864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cac24028-8470-4b5a-a9a4-e7cd85b08cf3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044274 Sep 18 06:33:12.558957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 872/1703] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:12 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:12.573712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:12.576461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:12.582817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.583299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.583730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.584403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.584849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.585266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.613429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-53fbf405-59b9-48f3-80f2-b6c0d2896108 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043649 Sep 18 06:33:12.613961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 832/1704] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:12 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:12.954661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:12.955220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:12.961180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.961656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.962087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:12.962799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:12.963249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:12.963669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.089458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:13.097083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e426ec4-7cec-4118-8fd4-1bbc5cb41e13 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146367 Sep 18 06:33:13.097644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 873/1705] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:12 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:13.404507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:13.406656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:13.408541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:13.413639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:13.414120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:13.414550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.415261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:13.415708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:13.416126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.421929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:13.422534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:13.428567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:13.429071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:13.429497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.430187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:13.430675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:13.431104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.447505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-45fb543d-4026-4192-8dda-9e54025b351b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047417 Sep 18 06:33:13.448074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 833/1706] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:13 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:13.567886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:13.570849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:13.573222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:13.577033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:13.578463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:13.579654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a4dd350d-a35f-4845-bf9b-b07b99739978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.162342 Sep 18 06:33:13.580025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:13.580513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:13.580625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 874/1707] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:13 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:13.580949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.581644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:13.582103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:13.582527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.612113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5d679f61-7f07-4a89-a83c-a12686352a98 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045245 Sep 18 06:33:13.612661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 834/1708] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:13 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:13.625450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:13.627832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:13.633661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:13.634142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:13.634593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.635274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:13.635721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:13.636141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:13.665896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-78c94f90-add9-4209-9f9e-a5ce1ba976ab tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044521 Sep 18 06:33:13.666369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 875/1709] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:13 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:14.119444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:14.120056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:14.126745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.127229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.127660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.128367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.128820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.129239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.257156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:14.265015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-53567d8b-cd27-4c7d-8a50-9152dd3a8bd0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149506 Sep 18 06:33:14.265576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 835/1710] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:14 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:14.461495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:14.463673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:14.469711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.470189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.470650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.471339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.471786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.472206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.502306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-50d65dcf-a3dd-44fd-a531-bb6fb24604b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044876 Sep 18 06:33:14.502898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 876/1711] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:14.604916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:14.605454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:14.612331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.612806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.613241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.613923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.614373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.614820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.625104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:14.627311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:14.633614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.634091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.634538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.635255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.635708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.636156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.665664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0180400-5ee4-44dc-80cd-2ea388ad8f85 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045180 Sep 18 06:33:14.666229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 877/1712] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:14.676919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:14.679240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:14.684988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.685467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.685902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.686614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:14.687075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:14.687498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:14.717798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d898d284-98c9-4141-b579-c080b4f3d40c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044482 Sep 18 06:33:14.718374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 878/1713] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:14 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:14.742848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:14.751623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:14.753078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:14.754257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-121abccc-89b7-4804-9094-c30a66e547ca tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153324 Sep 18 06:33:14.754842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 836/1714] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:14 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:15.289994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:15.290629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:15.296633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.297109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.297537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.298213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.298687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.299102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.432713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:15.440968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-700a1ff1-9496-4460-9109-0332869f4a5d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154904 Sep 18 06:33:15.441539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 879/1715] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:15 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:15.515521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:15.518876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:15.525267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.525753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.526189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.526906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.527365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.527791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.556159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f05ce950-2581-4bcb-b55f-dc333ec12cdd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045267 Sep 18 06:33:15.556642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 837/1716] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:15.678531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:15.680726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:15.687342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.687820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.688250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.688950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.689397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.689845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.726519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-711beb3a-3807-4649-bd40-0dbd2171ccbc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051784 Sep 18 06:33:15.727112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 880/1717] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:15 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:15.729956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:15.732788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:15.739206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.739689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.740127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.740818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.741265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.741688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.770983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b3b21f28-4dac-4048-8ff2-32dfc7b2cb82 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044916 Sep 18 06:33:15.771476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 838/1718] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:15 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:15.779594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:15.780232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:15.786145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.786653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.787086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.787765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:15.788211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:15.788630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:15.914810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:15.923285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:15.924704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:15.925858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-390ee6cf-dc27-45fb-928e-02a19430bedd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150770 Sep 18 06:33:15.926414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 881/1719] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:15 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:16.464486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:16.465099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:16.471259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.471744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.472175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.472853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.473301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.473721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.568960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:16.571176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:16.577088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.577569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.578000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.578711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.579162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.579582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.604333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:16.609508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dbd13b7b-8fd9-4dc1-8bc8-f5eb41079bbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044476 Sep 18 06:33:16.610190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 882/1720] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:16.612597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-591ca07a-4d1f-4361-850a-ca49a192a934 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151957 Sep 18 06:33:16.613154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 839/1721] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:16 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:16.739863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:16.741938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:16.749113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.749593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.750352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.750733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.751183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.751612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.781707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-108184d4-00b7-4481-85b8-5d03965de3ad tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046709 Sep 18 06:33:16.782277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 883/1722] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:16.782729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:16.784903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:16.791279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.791755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.792196 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.792893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.793354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.793790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.824101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a77bac10-01a9-4742-9304-1d30cfefbb58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045451 Sep 18 06:33:16.824654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 840/1723] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:16 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:16.951012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:16.951614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:16.957467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.957941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.958372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:16.959093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:16.959540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:16.959956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.084163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:17.093071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:17.094508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:17.095680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cf4c8bb1-054e-45d5-b4d5-84c30b8d1276 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.148618 Sep 18 06:33:17.096233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 884/1724] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:16 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:17.623060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:17.625234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:17.631424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:17.631912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:17.632349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.633044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:17.633496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:17.633920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.636346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:17.636957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:17.643488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:17.643964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:17.644398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.645081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:17.645531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:17.645951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.663799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-374fd9d8-d964-47c0-891f-64431b252fec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045493 Sep 18 06:33:17.664352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 841/1725] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:17.772458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:17.780249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6af41b19-4f9f-4c99-8596-ed5dce943f98 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147995 Sep 18 06:33:17.780811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 885/1726] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:17 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:17.793385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:17.795738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:17.802285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:17.802815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:17.803256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.803953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:17.804409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:17.804842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.835972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:17.836447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5f90eef2-7ad5-4745-8980-bce9364f3a2a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047032 Sep 18 06:33:17.837015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 842/1727] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:17 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:17.838201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:17.844127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:17.844608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:17.845051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.845745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:17.846198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:17.846647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:17.876895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c3007249-abc7-4b1f-9eda-7545b6b59b7c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044885 Sep 18 06:33:17.877472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 886/1728] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:17 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:18.121010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:18.121794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:18.128262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.128744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.129184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.129876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.130326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.130785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.258345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:18.267010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:18.268463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:18.269616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a2365f25-dc08-4b6b-b7a6-34e537a57f88 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152672 Sep 18 06:33:18.270171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 843/1729] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:18 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:18.676630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:18.678795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:18.684714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.685219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.685655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.686335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.686811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.687233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.715961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ada456c6-16b5-4609-af0a-6e3459bd70e6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043262 Sep 18 06:33:18.716935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 887/1730] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:18.803120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:18.803761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:18.809777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.810257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.810718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.811405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.811859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.812284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.850607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:18.852821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:18.858753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.859232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.859666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.860352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.860798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.861219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.891985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bed4d05b-8e5d-4931-85c7-cccb61a802cc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045659 Sep 18 06:33:18.896464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 888/1731] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:18 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:18.897158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:18.899286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:18.905588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.906067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.906496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.907331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:18.907782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:18.908203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:18.911824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:18.937227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3deedf46-f262-4ab6-b4dd-050771a935bf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043670 Sep 18 06:33:18.937790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 889/1732] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:18 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:18.942533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:18.945870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:18.951294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e9ea963a-9968-4435-85e6-6aa890ef0bfa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151350 Sep 18 06:33:18.951837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 844/1733] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:18 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:19.294447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:19.295084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:19.301755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.302270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.302692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.303377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.303823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.304237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.439403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:19.448519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:19.449962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:19.452299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d957ace8-bb90-45fe-b31b-349fe1e410b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.161666 Sep 18 06:33:19.453166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 890/1734] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:19 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:19.728758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:19.730996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:19.737877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.738353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.738829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.739515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.739966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.740384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.771018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a10027f0-df84-46de-99b3-d176bf4d61f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046172 Sep 18 06:33:19.771570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 845/1735] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:19.905367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:19.907534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:19.913876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.914359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.914828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.915509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.915957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.916378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.945781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6bb6fc71-307b-4b29-ac93-93da6609812a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044895 Sep 18 06:33:19.946351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 891/1736] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:19 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:19.949391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:19.951640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:19.957944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.958423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.958896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.959589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.960040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.960468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.974532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:19.975249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:19.981127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.981547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.982638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.983336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:19.983794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:19.984216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:19.992805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-85a3825c-a088-46fd-b6b6-35a92af1d9e7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047300 Sep 18 06:33:19.993388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 846/1737] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:19 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:20.112025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:20.119666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2fe38d36-0c2f-4188-b996-8c79bde45c8f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148962 Sep 18 06:33:20.120236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 892/1738] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:19 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:20.478158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:20.478836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:20.485053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:20.485528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:20.485957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:20.486667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:20.487117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:20.487540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:20.614606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:20.623770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:20.625203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:20.626359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-02c690cc-ad12-4793-903b-0ed368e499e6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152116 Sep 18 06:33:20.626942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 847/1739] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:20 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:20.783648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:20.785772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:20.791834 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:20.792316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:20.792754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:20.793449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:20.793905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:20.794330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:20.823499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d3cc78b2-a843-49f7-8f7f-faf88a3d7b00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043936 Sep 18 06:33:20.824054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 893/1740] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:20 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:20.958124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:20.960342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:20.966747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:20.967231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:20.967666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:20.968346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:20.968801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:20.969224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:20.998877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7956329e-a0f4-444a-a79b-681942652c30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044713 Sep 18 06:33:20.999422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 848/1741] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:20 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:21.005978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:21.008148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:21.014502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:21.015013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:21.015445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:21.016121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:21.016570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:21.016990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:21.046339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-288ad5e5-897c-4c02-9ab5-0a5080445744 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044175 Sep 18 06:33:21.046845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 894/1742] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:21 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:21.143004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:21.143617 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:21.149547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:21.150032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:21.150465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:21.151176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:21.151626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:21.152050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:21.278304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:21.285953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3ed2c274-291d-41f0-b8a0-09dc99b5e60d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146862 Sep 18 06:33:21.286509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 849/1743] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:21 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:21.651826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:21.652669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:21.659256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:21.659734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:21.660169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:21.660847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:21.661296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:21.661715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:21.794682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:21.803140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:21.804583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:21.805784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aed74ab0-a755-4ed9-b105-ab13a61442e4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157933 Sep 18 06:33:21.806345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 895/1744] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:21 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:21.836596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:21.838807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:21.845503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:21.845990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:21.846426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:21.847153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:21.847610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:21.848035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:21.877043 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d7fd717a-6322-473e-813b-45f4967ea408 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044399 Sep 18 06:33:21.877596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 850/1745] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:22.018250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:22.018250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:22.023359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.023843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.024278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.025034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.025485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.025913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.058845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:22.061043 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:22.061788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-28dabe61-2f2b-4b5e-accd-5ae38dd4d61a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053598 Sep 18 06:33:22.062349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 896/1746] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:22 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:22.067476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.067956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.068390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.069074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.069520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.069936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.099538 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-468bb892-dc8c-45c9-8231-5af27ae1efff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044616 Sep 18 06:33:22.100084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 851/1747] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:22 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:22.309147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:22.310173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:22.316047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.316521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.316950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.317626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.318072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.318491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.458127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:22.465973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6b574395-dae5-4187-ba82-dec341de0fae tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.160673 Sep 18 06:33:22.466549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 897/1748] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:22 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:22.830536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:22.831168 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:22.837265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.837743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.838173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.838877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.839330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.839775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.888534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:22.890738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:22.896656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.897137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.897578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.898949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:22.899410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:22.899837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:22.930426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d7b7e395-66aa-43f3-9db1-343f569377b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045813 Sep 18 06:33:22.931036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 898/1749] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:22 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:22.966959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:22.975473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:22.976912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:22.978072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b5e1fd21-fdb7-47f1-9bab-011bc082bbb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151456 Sep 18 06:33:22.978628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 852/1750] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:22 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:23.073131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:23.075327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:23.081717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:23.082145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:23.082601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:23.083289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:23.083743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:23.084171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:23.112119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:23.114178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ca37e8c8-1ff3-46ce-907e-287b3033c542 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044994 Sep 18 06:33:23.114305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:23.114760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 899/1751] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:23 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:23.121374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:23.121861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:23.122388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:23.123108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:23.123554 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:23.123973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:23.152633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4e383cf9-6b70-4aab-be85-849117386a0e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044663 Sep 18 06:33:23.153708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 853/1752] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:23 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:23.488431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:23.489046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:23.495045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:23.495531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:23.495969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:23.496911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:23.497400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:23.497860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:23.629054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:23.636824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-10e90a33-4638-4357-94bd-be6137bac8e9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152287 Sep 18 06:33:23.637415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 900/1753] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:23 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:23.942847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:23.945018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:23.951518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:23.951995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:23.952425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:23.953100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:23.953547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:23.953967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:23.982720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8882f133-83fc-4cba-a20d-c23a85b06265 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043877 Sep 18 06:33:23.983256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 854/1754] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:24.003831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:24.004438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:24.010311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:24.010848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:24.011288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:24.011975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:24.012429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:24.012859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:24.127609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:24.129969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:24.136190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:24.136674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:24.136923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:24.137359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:24.137807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:24.138257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:24.138711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:24.145765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:24.147268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:24.148439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6c52cda4-9aa8-44eb-bc5c-571eca88ee0d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.148529 Sep 18 06:33:24.148998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 901/1755] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:23 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:24.165170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:24.167645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-91acedd9-fee0-40aa-81ae-d241aabfd3dd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043902 Sep 18 06:33:24.167808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:24.168397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 855/1756] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:24 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:24.170196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:24.175650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:24.176135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:24.176573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:24.177269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:24.177751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:24.178177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:24.208167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c8b2a42c-9a2d-4d82-abc2-57055c6e5061 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046856 Sep 18 06:33:24.208662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 902/1757] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:24 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:24.660259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:24.660878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:24.666858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:24.667340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:24.667778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:24.668461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:24.669511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:24.669940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:24.797307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:24.804988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4993917d-3273-4c65-8506-561844d24850 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148582 Sep 18 06:33:24.805545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 856/1758] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:24 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:24.994141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:24.996681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:25.002631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.003121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.003560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.004284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.004742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.005175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.034683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-50a35317-bbc1-4319-9088-cb8813012266 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044487 Sep 18 06:33:25.035232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 903/1759] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:24 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:25.173948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:25.174567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:25.180332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:25.181044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.181528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.181963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.182503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:25.182682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.183136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.183558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.189265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.189741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.190169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.190882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.191330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.191752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.221119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3b4e0eef-cf8a-458b-830a-138df79ae63d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044612 Sep 18 06:33:25.221689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 904/1760] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:25 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:25.226348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:25.228506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:25.234227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.234738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.235173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.235860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.236537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.236964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.262295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:25.268250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b46e90fc-00bb-41fa-8e8b-ab52c08ca09d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045604 Sep 18 06:33:25.268816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 905/1761] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:25 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:25.314189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:25.322767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:25.324222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:25.326271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8866403b-dd2e-4481-b6c0-a90fe3e858d8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155337 Sep 18 06:33:25.327290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 857/1762] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:25 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:25.827836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:25.828443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:25.834497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.835006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.835437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.836111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:25.836573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:25.836975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:25.964466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:25.972903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cb4ae5b2-046b-4582-a091-58810d12c964 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148947 Sep 18 06:33:25.973467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 906/1763] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:25 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:26.048159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:26.050326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:26.056296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:26.056779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:26.057219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:26.057906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:26.058360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:26.058819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:26.087209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f0a26b21-05bf-4b53-aab2-3f4d435c0ebd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043719 Sep 18 06:33:26.087749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 858/1764] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:26.233986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:26.236119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:26.242394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:26.242900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:26.243332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:26.244012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:26.244460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:26.244882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:26.273733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e91576f2-0eba-4813-84a3-ecf99d6601aa tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044374 Sep 18 06:33:26.274274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 907/1765] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:26 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:26.280432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:26.282613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:26.288918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:26.289396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:26.289830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:26.290515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:26.290993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:26.291422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:26.319898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3eafea80-9fa6-456b-8643-55c1e5eb3556 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044000 Sep 18 06:33:26.320459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 859/1766] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:26 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:26.352671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:26.353273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:26.359230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:26.359702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:26.360127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:26.360807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:26.361250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:26.361665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:26.490837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:26.499571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:26.500997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:26.502149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-68eca029-580f-4adf-a932-1005833f9b06 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153405 Sep 18 06:33:26.502728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 908/1767] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:26 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:26.997427 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:26.998048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:27.004172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.004684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.005122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.005840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.006293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.006751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.099261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:27.101441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:27.107855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.108333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.108764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.109448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.109897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.110316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.134229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:27.140060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-257f51eb-734a-4c1a-9088-8ed9b4f01006 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044743 Sep 18 06:33:27.140629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 909/1768] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:27.141997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a21cbcdc-c2d0-45ea-b8b3-448f77d5834b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149249 Sep 18 06:33:27.142547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 860/1769] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:27.286480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:27.288571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:27.295345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.295818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.296254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.296958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.297402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.297848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.327242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-acf47062-ee6f-4e1d-a48d-6d6d4820f3ce tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044605 Sep 18 06:33:27.327785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 910/1770] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:27 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:27.332171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:27.334336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:27.341031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.341513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.341949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.342764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.343471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.344206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.363264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:27.377654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-efc4d587-cef6-44e2-b1d6-4b8d905131df tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049537 Sep 18 06:33:27.378236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 861/1771] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:27 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:27.526714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:27.527343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:27.533770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.534253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.534704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.535384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:27.535830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:27.536251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:27.665039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:27.674139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:27.675595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:27.676744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b5002857-21be-46ca-8500-0e02752370c3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153998 Sep 18 06:33:27.677293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 911/1772] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:27 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:28.151889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:28.154028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:28.160290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.160769 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.161206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.161888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.162340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.162797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.164646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:28.165608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:28.171522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.172007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.172441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.173128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.173579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.174002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.192624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ad946f70-8cf8-4f48-b72a-4e80e0d58ca6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044688 Sep 18 06:33:28.193175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 862/1773] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:28 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:28.300162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:28.307741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aed92354-ea2a-47ae-bdfc-6e838681fe4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146877 Sep 18 06:33:28.308303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 912/1774] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:28 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:28.339445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:28.341652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:28.347986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.348467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.348902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.349585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.350078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.350506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.379126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a3dc52e5-54f6-45d3-ba6e-56b7089fe8b5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043591 Sep 18 06:33:28.380567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 863/1775] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:28 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:28.389775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:28.391964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:28.397835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.398309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.398774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.399456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.399902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.400322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.428780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-852f1ef2-717a-4199-89fa-4fe2740d3bec tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043437 Sep 18 06:33:28.429262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 913/1776] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:28 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:28.701771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:28.702327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:28.708373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.708861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.709297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.709984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:28.710439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:28.710922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:28.837559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:28.846120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:28.847604 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:28.848764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b4142e80-cf7b-419f-a111-3fad530ad1c1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151084 Sep 18 06:33:28.849310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 864/1777] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:28 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:29.204970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:29.207211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:29.213051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.213535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.213969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.214678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.215137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.216092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.245743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5e1b2372-3869-47ed-8144-ad1a9de7c63c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044639 Sep 18 06:33:29.246571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 914/1778] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:29.331559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:29.332174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:29.338075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.338562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.339037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.339733 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.340305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.340599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.393084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:29.396006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:29.402719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.403283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.404375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.405139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.405681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.406151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.437355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-216919a7-9189-42ae-8d45-c928d060a0b7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049062 Sep 18 06:33:29.437916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 915/1779] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:29 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:29.442758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:29.444889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:29.451104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.451770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.452382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.453230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.453833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.454411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.484916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:29.485488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-307707b2-b83e-4260-87b2-ac0bdab74c3d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046350 Sep 18 06:33:29.486065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 916/1780] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:29 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:29.492704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2a656428-c072-4675-819e-c054d3835736 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.164955 Sep 18 06:33:29.493259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 865/1781] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:29 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:29.874320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:29.874969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:29.880974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.881463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.881916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:29.882604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:29.883084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:29.883496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.009461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:30.018507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:30.019976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:30.021141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-03a5b029-86f2-4934-bc21-457955a7d1be tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151025 Sep 18 06:33:30.021705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 917/1782] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:29 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:30.258410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:30.260608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:30.267027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:30.267501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:30.267932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.268610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:30.269060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:30.269477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.298444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4d27252b-5fbe-4fda-a118-48d7f09f67f6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044008 Sep 18 06:33:30.299020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 866/1783] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:30 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:30.449474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:30.452507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:30.459172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:30.459656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:30.460094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.460791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:30.461243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:30.461664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.490087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-70d73402-c71e-4995-9bdd-6d94a4586e87 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044611 Sep 18 06:33:30.490846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 918/1784] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:30 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:30.497575 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:30.500232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:30.506787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:30.507265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:30.507697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.508467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:30.508854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:30.509271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.516341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:30.516957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:30.522894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:30.523376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:30.523821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.524514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:30.524972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:30.525399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:30.540964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-23c2f92d-21d8-4ff8-8f35-85cd4e5322a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047524 Sep 18 06:33:30.541519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 867/1785] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:30 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:30.653947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:30.661703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5dd2700d-6d74-4500-9774-c3a565b65d04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149787 Sep 18 06:33:30.662275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 919/1786] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:30 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:31.050177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:31.050812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:31.058600 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.059079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.059511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.060192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.060640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.061645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.184862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:31.195596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:31.197020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:31.198170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3263829f-364a-4390-9e0d-000d15f54050 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155511 Sep 18 06:33:31.198747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 868/1787] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:31 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:31.310833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:31.312909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:31.318915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.319397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.319833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.320574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.321023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.321510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.354393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-def410b9-fdb6-4309-aad9-cdd870d05aaa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047577 Sep 18 06:33:31.355271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 920/1788] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:31.502747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:31.504959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:31.511375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.511863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.512301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.512993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.513444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.514378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.543436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fb8697b8-dcec-4f99-bf2f-50a8ec0249ee tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044621 Sep 18 06:33:31.544000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 869/1789] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:31 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:31.553184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:31.555348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:31.561651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.562078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.562518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.563222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.563674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.564098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.593735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-39b7acd9-abc3-4356-b08b-fb03ac38113c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044481 Sep 18 06:33:31.594281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 921/1790] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:31 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:31.684292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:31.684912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:31.691001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.691492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.691941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.692628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:31.693086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:31.693518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:31.820975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:31.829250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8c7c66bf-19a7-48dc-a7e9-eca61853e046 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148896 Sep 18 06:33:31.829815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 870/1791] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:31 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:32.223079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:32.223710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:32.229648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.230126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.230560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.231258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.231707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.232125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.247170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:32.362063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:32.368002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:32.370193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:32.371069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:32.372519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:32.374319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-60eeb526-5ce8-4a3e-9403-ecd3b6a6f97a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155154 Sep 18 06:33:32.374908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 922/1792] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:32 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:32.376925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.377407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.377841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.378521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.379001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.379420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.408284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1672e1fc-4393-498c-9d7d-a90b1a1eb3c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044169 Sep 18 06:33:32.408830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 871/1793] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:32 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:32.555751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:32.557886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:32.564763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.565240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.565673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.566349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.566831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.567251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.596178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-76f25ade-3403-4149-a1ed-bd6b4d28ee30 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044375 Sep 18 06:33:32.596722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 923/1794] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:32 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:32.605394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:32.607566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:32.613806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.614288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.614774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.615460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.615912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.616341 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.646109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-75421368-3d65-473d-9c91-73a25778fb26 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044628 Sep 18 06:33:32.646682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 872/1795] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:32 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:32.852339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:32.852900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:32.859423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.859900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.860361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.861037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:32.861486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:32.861906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:32.987587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:32.995211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ae8fb797-a167-4088-93cc-f30766d8dd58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146907 Sep 18 06:33:32.996044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 924/1796] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:32 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:33.399074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:33.399685 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:33.405806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:33.406286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:33.406751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:33.407435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:33.407883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:33.408308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:33.421213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:33.423390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:33.429264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:33.429743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:33.430174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:33.430895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:33.431345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:33.431764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:33.463268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c048fecd-a5b4-4691-8e5b-4a2cf42a8773 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045875 Sep 18 06:33:33.464161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 925/1797] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:33 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:33.538063 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:33.546560 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:33.548031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:33.549177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-36a37634-6a61-4844-9961-0e209831b0f0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154029 Sep 18 06:33:33.549726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 873/1798] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:33 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:33.608608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:33.610816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:33.617177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:33.617662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:33.618097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:33.618809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:33.619262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:33.619686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:33.649324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8befe80a-28ee-48c2-b4a4-06c218cdda80 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044687 Sep 18 06:33:33.649810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 926/1799] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:33 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:33.658322 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:33.660533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:33.666888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:33.667362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:33.667793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:33.668469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:33.668920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:33.669339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:33.699003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d10386f6-6ecf-469d-86f4-09301329fcd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044615 Sep 18 06:33:33.699557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 874/1800] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:33 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:34.018639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:34.019176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:34.025113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.025592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.026028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.026759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.027197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.027619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.152867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:34.161032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d4d190d8-b390-455c-9ec7-eee8d15d2c08 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146438 Sep 18 06:33:34.161598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 927/1801] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:34 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:34.475037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:34.477165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:34.483140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.483614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.484047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.484728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.485179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.485602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.514688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a953e4a2-5bd0-4fa9-b865-3ab85b0757a8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043620 Sep 18 06:33:34.515226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 875/1802] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:34.574514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:34.575143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:34.581714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.582188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.582647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.583331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.583777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.584198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.661490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:34.663681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:34.669978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.670461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.670937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.671629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.672077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.672567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.704029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-80f2b3f6-c94b-41ac-8bd8-628c715a6e34 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046472 Sep 18 06:33:34.708978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 876/1803] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:34 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:34.711144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:34.713376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:34.715333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:34.720276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.720768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.721205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.721912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:34.722413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:34.722933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:34.725344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:34.728074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:34.729262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-86860206-20fa-4dca-b91d-621dc41bd5bd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.158601 Sep 18 06:33:34.730163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 928/1804] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:34 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:34.755164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-457bf8ad-20b7-42f3-95b6-d6e6222d0f58 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047780 Sep 18 06:33:34.755760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 877/1805] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:34 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:35.184851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:35.186009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:35.191930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.192413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.192845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.193554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.194000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.194420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.336954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:35.345600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a3179d41-d597-419d-adee-a4a029510f41 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.164665 Sep 18 06:33:35.346167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 929/1806] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:35 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:35.527113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:35.529811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:35.535743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.536226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.536660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.537349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.537806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.538230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.541839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:35.567855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-56b8f42b-db22-4de9-955b-6ecae4320ac5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044815 Sep 18 06:33:35.568399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 878/1807] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:35.715941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:35.718107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:35.724466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.724937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.725364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.726038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.726481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.726928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.755086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:35.755698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:35.759130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7c37d13d-37bb-4d12-a4c2-db5508c0ac76 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047149 Sep 18 06:33:35.759778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 930/1808] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:35 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:35.762968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.763669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.764642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.765413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.765877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.766304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.770051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:35.772438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:35.778349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.778800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.779287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.779972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:35.780450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:35.780867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:35.809938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2702a99e-d277-4581-bb78-e1404c4ce73b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046356 Sep 18 06:33:35.811069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 931/1809] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:35 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:36.143685 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:36.152290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:36.153761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:36.154995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-09586e9c-c981-4fab-b417-ea9a12baad3b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.403769 Sep 18 06:33:36.155550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 879/1810] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:35 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:36.371687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:36.372375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:36.379649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:36.380381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:36.381039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:36.382097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:36.382826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:36.383491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:36.514011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:36.521736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e42919d6-8b8c-44c9-a795-f253b5cb5ad1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155066 Sep 18 06:33:36.522302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 932/1811] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:36 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:36.580163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:36.582310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:36.588213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:36.588691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:36.589122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:36.589797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:36.590245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:36.590690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:36.619853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ba405062-45ec-45e7-9435-1e512b278b9f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043675 Sep 18 06:33:36.620388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 880/1812] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:36.773549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:36.775659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:36.782763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:36.783239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:36.783668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:36.784344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:36.784789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:36.785209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:36.813409 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e4880d33-2fed-4f19-b85f-a25139792d7a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044011 Sep 18 06:33:36.813949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 933/1813] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:36 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:36.822702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:36.824882 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:36.831772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:36.832254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:36.832690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:36.833375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:36.833829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:36.834254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:36.863468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-290a4edf-76ab-4f89-8317-c04e71515735 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044698 Sep 18 06:33:36.864011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 881/1814] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:36 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:37.180255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:37.180854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:37.187295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.187773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.188204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.188880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.189327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.189750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.314649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:37.323227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:37.324666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:37.325830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b3dcd055-6ea0-45d5-8ce4-03b46ab46ec6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149511 Sep 18 06:33:37.327220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 934/1815] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:37 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:37.545871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:37.546483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:37.552545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.553028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.553465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.554149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.554628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.555055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.633068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:37.635280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:37.641313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.641791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.642224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.642939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.643388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.643813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.673418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d98a96e2-6298-4f66-97d3-f59119a23a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044499 Sep 18 06:33:37.673984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 935/1816] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:37 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:37.683683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:37.693828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c6a9e437-4493-4b18-98e4-95635e05ef8a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152031 Sep 18 06:33:37.695068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 882/1817] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:37 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:37.825457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:37.827636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:37.834004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.834484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.834957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.835645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.836099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.836525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.866049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8435126c-3d3b-4556-8919-eb2153d790e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044618 Sep 18 06:33:37.866620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 936/1818] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:37 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:37.875957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:37.878093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:37.880339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:37.885618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.886096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.886525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.887302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:37.887679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:37.888098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:37.917122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e9d8fb98-7626-4c55-8533-d796bb0d5790 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045138 Sep 18 06:33:37.917667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 883/1819] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:37 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:38.351309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:38.351911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:38.357884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.358361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.358833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.359515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.359962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.360381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.491129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:38.499989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:38.501466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:38.502663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-659f27a9-e29f-4cc1-aa3e-c2e3933fdf74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155268 Sep 18 06:33:38.503220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 937/1820] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:38 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:38.686112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:38.688260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:38.694132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.694637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.695076 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.695782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.696233 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.696652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.718218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:38.718891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:38.724717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.725197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.725631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.726162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ef0df8b9-cc30-4482-b8ed-bad0557dfe10 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044451 Sep 18 06:33:38.726294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.726902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 884/1821] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:38 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:38.727021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.727192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.857428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:38.865086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-22ae4513-bee1-4d13-b218-8bb32156a908 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150712 Sep 18 06:33:38.865724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 938/1822] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:38 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:38.879437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:38.881574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:38.887853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.888976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.889419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.890101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.890551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.891003 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.918778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9b519a8e-150a-40f4-86de-6edc25201aef tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043285 Sep 18 06:33:38.919315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 885/1823] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:38 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:38.929243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:38.931425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:38.937888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.938311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.938789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.939475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:38.939928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:38.940355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:38.943962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:38.969970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-280ad42d-7812-4eb4-a766-606b077c2787 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044602 Sep 18 06:33:38.970517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 939/1824] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:38 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:39.528125 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:39.528735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:39.535201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.535686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.536124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.536810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.537265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.537696 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.661988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:39.671499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:39.672954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:39.674163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6af56b3d-40cb-4cea-800f-668f38de974f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149962 Sep 18 06:33:39.674747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 886/1825] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:39 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:39.738446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:39.740624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:39.746909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.747389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.747829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.748516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.748969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.749396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.778503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e1d40397-cf05-47fc-a287-e54ed5ad7015 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044018 Sep 18 06:33:39.779746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 940/1826] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:39 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:39.888710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:39.889331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:39.896321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.896803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.897240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.897930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.898387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.898906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.931075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:39.933162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:39.939586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.940072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.940509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.941211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.941663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.942086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.972559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-47f730cb-b6ce-4780-a9b4-099579a52d1d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045395 Sep 18 06:33:39.973135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 941/1827] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:39 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:39.981162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:39.983308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:39.989360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.989838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.990275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:39.990991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:39.991444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:39.991865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:40.022277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4bbc36fd-f2e2-4c53-b296-8f4cae645a24 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044984 Sep 18 06:33:40.022885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 942/1828] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:39 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:40.036481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:40.044270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-02dc5cb9-e939-4ffe-add9-fbe59577736d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.159440 Sep 18 06:33:40.044823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 887/1829] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:39 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:40.699727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:40.700337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:40.706312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:40.706830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:40.707271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:40.707961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:40.708414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:40.708842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:40.791548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:40.793949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:40.800556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:40.801049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:40.801485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:40.802180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:40.802671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:40.803101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:40.833042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2b10ed44-6a78-43dc-b971-2fa242f3ad8d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045342 Sep 18 06:33:40.833614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 888/1830] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:40.835846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:40.842480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:40.844442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:40.845602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-694653f1-ea31-415f-b738-5d6b4d5d53e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149856 Sep 18 06:33:40.846155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 943/1831] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:40 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:40.985599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:40.987816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:40.994108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:40.994594 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:40.995037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:40.995981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:40.996428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:40.996844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.024826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-bdc905d3-0a65-4974-a5f3-84811a21ca94 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043680 Sep 18 06:33:41.025359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 889/1832] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:40 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:41.035749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:41.037883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:41.044382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:41.044867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:41.045306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.046700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:41.047159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:41.047591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.067266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:41.067876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:41.073736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:41.074217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:41.074680 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.075369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:41.075816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:41.076236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.076954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ec9132a6-662f-4000-90c6-e9d30d366d69 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045169 Sep 18 06:33:41.077525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 944/1833] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:41 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:41.206237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:41.214008 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-12e36bb9-9ce0-46e5-babd-f783e4b913e0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150513 Sep 18 06:33:41.214571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 890/1834] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:41 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:41.845621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:41.847831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:41.854093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:41.854572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:41.855133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.855812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:41.856261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:41.856685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.871102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:41.871862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:41.878198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:41.878723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:41.879166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.879865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:41.880323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:41.880752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:41.886737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d23f4062-af38-446c-a173-cc92395d926c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045048 Sep 18 06:33:41.887304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 945/1835] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:42.005657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:42.014216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:42.016113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:42.017272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ce3a69d9-4ead-4c56-89cc-df9a12381775 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150319 Sep 18 06:33:42.017823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 891/1836] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:41 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:42.036288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:42.038456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:42.044753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:42.045232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:42.046195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:42.046904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:42.047419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:42.047783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:42.076389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-11df7a2f-7f24-484e-99cb-3c121e7a06cf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044014 Sep 18 06:33:42.076965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 946/1837] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:42 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:42.089134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:42.091309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:42.097346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:42.097836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:42.098274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:42.098990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:42.099463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:42.099889 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:42.129632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3f3fbb48-4cd2-45fe-a2ae-0cf10fe8911f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044436 Sep 18 06:33:42.130180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 892/1838] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:42 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:42.237300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:42.237904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:42.243828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:42.244299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:42.244725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:42.245401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:42.246116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:42.246535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:42.374454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:42.382333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a6a65fde-6ede-4ef6-8d34-e74b2d44adc3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149350 Sep 18 06:33:42.382954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 947/1839] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:42 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:42.900266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:42.902406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:42.909020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:42.909501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:42.909938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:42.910650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:42.911108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:42.911538 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:42.939777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3008de2e-3560-42f0-a74e-f842993f9518 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043590 Sep 18 06:33:42.940317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 893/1840] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:43.041791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:43.042398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:43.048737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.049211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.049642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.050324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.050798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.051219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.088440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:43.090569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:43.096925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.097402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.097832 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.098519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.098995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.099417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.124205 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:43.129859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c90d1eff-5c63-42de-8445-b94121b4aeba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045257 Sep 18 06:33:43.130418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 894/1841] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:43 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:43.141648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:43.143930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:43.150059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.150678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.151037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.151739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.152198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.152627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.181716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:43.183892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-36b81c62-f22c-49fc-8e57-3f58e34231f3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046595 Sep 18 06:33:43.184453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 895/1842] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:43 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:43.190880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:43.192366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:43.193648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1ae40196-2453-47b3-be4c-9d89775eb129 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155750 Sep 18 06:33:43.194208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 948/1843] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:43 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:43.405260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:43.405831 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:43.411860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.412338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.412770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.413447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.413895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.414311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.544509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:43.552806 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ecc1252b-a59a-4985-9de1-a2932ece078b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151422 Sep 18 06:33:43.553368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 896/1844] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:43 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:43.952739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:43.955000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:43.960948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.961420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.961847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.962532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:43.963005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:43.963424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:43.991568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9d7ad372-732a-49cf-862f-db94482b19af tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043482 Sep 18 06:33:43.992107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 949/1845] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:44.142832 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:44.144985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:44.151474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:44.151928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:44.152361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:44.153041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:44.153491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:44.153912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:44.182623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f52b8452-283c-497b-a57b-abd64e9398ed tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044368 Sep 18 06:33:44.183172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 897/1846] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:44 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:44.196629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:44.198797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:44.205121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:44.205592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:44.206022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:44.206730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:44.207179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:44.207596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:44.218346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:44.219049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:44.224813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:44.225293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:44.225727 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:44.226413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:44.226893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:44.227321 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:44.237777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e6661084-806c-436f-9905-7c03745af375 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045038 Sep 18 06:33:44.238339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 950/1847] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:44 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:44.356797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:44.365792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:44.367259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:44.368406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a835e829-0dba-43b3-a6c4-18cf453921ec tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153841 Sep 18 06:33:44.368950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 898/1848] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:44 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:44.576430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:44.577067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:44.583210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:44.583686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:44.584119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:44.584805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:44.585256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:44.585679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:44.712800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:44.721853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4106dc51-43fa-4c70-a926-396ab5170e3f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149240 Sep 18 06:33:44.722368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 951/1849] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:44 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:45.003481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:45.005511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:45.011497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.011974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.012413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.013091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.013540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.013961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.043179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4832acb7-a8f1-40c4-928f-4e0e073797ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043899 Sep 18 06:33:45.043710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 899/1850] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:45.195710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:45.197851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:45.199721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:45.205230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.205708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.206137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.206852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.207301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.207718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.236808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6c0c8e8c-c35e-4262-8c12-0644e7c67dba tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045071 Sep 18 06:33:45.237356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 952/1851] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:45 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:45.249400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:45.251582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:45.257907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.258329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.258809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.259494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.259942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.260364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.289759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c2997dfd-3a2b-41b2-89b3-1078ca0842b8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044478 Sep 18 06:33:45.290312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 900/1852] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:45 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:45.394959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:45.395687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:45.401778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.402269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.402758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.403453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.403907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.404366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.532062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:45.540609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:45.542046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:45.543238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f26daea8-e77c-42a4-8a18-8eb4c6589994 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152327 Sep 18 06:33:45.543795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 953/1853] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:45 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:45.746132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:45.746807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:45.753915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.754391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.754845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.755728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:45.756175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:45.756592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:45.887793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:45.895556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4dd7b6d4-df61-4cdd-8dc6-8988f9a6198c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153798 Sep 18 06:33:45.896109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 901/1854] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:45 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:46.055577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:46.057754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:46.063682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.064160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.064593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.065282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.065732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.066165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.097962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f4b4d5e2-e192-4126-ac53-432459bf2fd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046227 Sep 18 06:33:46.098477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 954/1855] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:46.249561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:46.251746 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:46.257821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.258302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.258778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.259464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.259915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.260336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.288855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-12d82675-cf8a-4fe2-a74a-a707babbdc96 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043600 Sep 18 06:33:46.289398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 902/1856] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:46 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:46.301930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:46.304214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:46.310632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.311157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.311590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.312275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.312727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.313148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.337751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:46.343652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a4e583d6-1983-4ee3-a844-704b2d9f03bb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045703 Sep 18 06:33:46.344207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 955/1857] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:46 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:46.567319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:46.567934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:46.573856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.574339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.574816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.575503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.575957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.576416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.700820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:46.709827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:46.711297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:46.712458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-840194a8-869f-4315-85a3-5eb3997ae521 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149121 Sep 18 06:33:46.713046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 903/1858] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:46 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:46.918239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:46.918866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:46.924960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.925444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.925878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:46.926618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:46.927069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:46.927488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.057325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:47.065408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-51e06972-2b69-4bd9-adbd-64b8ca448f02 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151101 Sep 18 06:33:47.065966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 956/1859] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:46 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:47.110313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:47.112492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:47.118363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:47.118881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:47.119325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.120022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:47.120760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:47.121191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.150404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a5a599b6-48ad-4fae-b7ca-f0238b6f805b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044049 Sep 18 06:33:47.150977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 904/1860] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:47.301702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:47.303839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:47.310023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:47.310442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:47.311531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.312237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:47.312686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:47.313103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.341386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f0e94678-33e9-4a9d-ac10-3196e70f9f6d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043628 Sep 18 06:33:47.341985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 957/1861] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:47 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:47.355953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:47.358791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:47.365104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:47.365579 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:47.366010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.366742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:47.367191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:47.367611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.397735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-680c9fb0-cda8-4da1-bc42-aea17c709cee tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045639 Sep 18 06:33:47.398276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 905/1862] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:47 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:47.737326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:47.737958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:47.744880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:47.745633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:47.746068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.746801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:47.747258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:47.747683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:47.878539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:47.887452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:47.888857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:47.890091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6d68f60a-de93-44a0-8fac-92c4021f1a83 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156729 Sep 18 06:33:47.890681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 958/1863] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:47 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:48.088943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:48.089494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:48.096198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.096686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.097124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.097821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.098279 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.098740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.162962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:48.165197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:48.171167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.171644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.172083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.172764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.173372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.173646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.203404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-477b8672-1739-439c-ba70-52dd343b7c87 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044373 Sep 18 06:33:48.203973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 959/1864] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:48.229385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:48.237267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a484f32a-0eaf-4c7f-adce-21898cbb5109 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152352 Sep 18 06:33:48.237828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 906/1865] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:48 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:48.353367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:48.355461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:48.361819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.362350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.362812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.363490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.363937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.364355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.393818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-97f60e4d-9614-4ba4-ad8f-947e3702c13d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044434 Sep 18 06:33:48.394372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 960/1866] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:48 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:48.409216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:48.411443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:48.417740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.418221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.418693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.419377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.419830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.420254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.449214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9af22cbf-2a38-44e9-ba8b-4015c4fcc017 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043883 Sep 18 06:33:48.449948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 907/1867] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:48 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:48.916073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:48.916682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:48.923337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.923817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.924255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:48.924942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:48.925393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:48.925820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.051523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:49.060255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:49.061726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:49.062963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5f0892b3-5cf9-4f68-8acd-d2a77d1a71fa tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150847 Sep 18 06:33:49.063537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 961/1868] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:48 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:49.215641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:49.218304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:49.224472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:49.224955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:49.225390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.226079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:49.226535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:49.226993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.256493 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0247e9d9-b24b-4a30-bf11-84fb980ac5ce tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044779 Sep 18 06:33:49.257037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 908/1869] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:49.260235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:49.260843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:49.266804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:49.267301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:49.267723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.268437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:49.268891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:49.269314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.396230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:49.405353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3c50e112-6eef-4b47-8550-1337a2d6fb68 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148290 Sep 18 06:33:49.405955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:49.415499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 962/1870] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:49 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:49.415699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:49.415699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:49.415699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:49.415699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.421662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:49.421662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:49.421662 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.449268 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b3489009-b0c5-4563-94a9-981aef63ca32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047717 Sep 18 06:33:49.449838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 909/1871] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:49 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:49.461272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:49.463429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:49.470048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:49.470471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:49.470935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.471622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:49.472065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:49.472482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:49.501841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fb447e0e-b7d4-4026-9da1-1dee085658d0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044513 Sep 18 06:33:49.502402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 963/1872] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:49 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:50.087876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:50.088517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:50.094777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.095263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.095697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.096386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.096835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.097254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.223788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:50.232180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:50.234135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:50.235319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-07d79490-e9a5-4409-bf60-a2ee8372ffff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151380 Sep 18 06:33:50.235872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 910/1873] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:50 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:50.268587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:50.270766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:50.277117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.277600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.278037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.278747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.279201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.279624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.308527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b07eaf77-3f9e-4a3c-9e37-9151284c95f1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043857 Sep 18 06:33:50.309065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 964/1874] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:50.434026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:50.435091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:50.441729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.442238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.442704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.443382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.443828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.444249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.461516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:50.462751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:50.463953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:50.469924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.470406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.470878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.471577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.472030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.472454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.503111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-85606e64-7a16-48de-a272-0b5901d1a2bc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045698 Sep 18 06:33:50.503681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 965/1875] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:50 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:50.512921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:50.515200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:50.521242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.521723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.522160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.523188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:50.523391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:50.523819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:50.554672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a0d36f59-4dae-456d-b35f-f6ade9123938 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045377 Sep 18 06:33:50.556956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 966/1876] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:50 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:50.576673 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:50.584403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-feaa831e-c3d5-4398-8f06-90bb903c8cc8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154213 Sep 18 06:33:50.584959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 911/1877] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:50 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:51.260891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:51.261517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:51.267996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.268474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.268903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.269584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.270030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.270450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.321437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:51.323640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:51.331597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.332085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.332514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.333199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.333679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.334100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.365297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e685dbd2-184a-4a41-aaba-7ae330a189c0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047773 Sep 18 06:33:51.365861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 912/1878] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:51.406125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:51.415152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:51.416615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:51.417803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3408bf9d-815f-48b8-9277-59f231a2dc2f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.160871 Sep 18 06:33:51.418370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 967/1879] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:51 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:51.515529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:51.517626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:51.523544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.524021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.524453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.525132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.525580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.525998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.553751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7b76cb98-57f9-486a-81e7-256633b9a1e0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042931 Sep 18 06:33:51.554287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 913/1880] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:51 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:51.568083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:51.570218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:51.576124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.577267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.577693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.578372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.578916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.579339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.607609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:51.608242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:51.608539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-72eb9151-5036-4b93-9163-7de07bf6004f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044369 Sep 18 06:33:51.609104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 968/1881] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:51 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:51.614161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.614669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.615103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.615778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:51.616223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:51.616640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:51.746051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:51.754720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-25d0f350-2063-4016-9c92-ab9d0ce67ec3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150799 Sep 18 06:33:51.755434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 914/1882] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:51 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:52.378115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:52.380400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:52.387579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.388115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.388554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.389266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.389720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.390149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.427679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-45a1629c-3d4f-46ac-aa05-b3ee2651c7cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053688 Sep 18 06:33:52.428505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 969/1883] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:52 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:52.442286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:52.442935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:52.449407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.449902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.450325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.451035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.451983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.452408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.565430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:52.568255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:52.575663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.576636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.577071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.577756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.578197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.578637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.581791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:52.590391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:52.592432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:52.593618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-61521dcb-08b0-4dc3-8b60-02dadf31fae8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.155197 Sep 18 06:33:52.594208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 915/1884] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:52 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:52.608073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7041fefe-b6ac-4069-8ef9-4fd9c9487bfb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046599 Sep 18 06:33:52.608618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 970/1885] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:52 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:52.620852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:52.623219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:52.629284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.629711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.630145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.630846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.631298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.631721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.661296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b2281f9a-6a44-483b-9f69-91568a0daf00 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044470 Sep 18 06:33:52.661839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 916/1886] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:52 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:52.778382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:52.779033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:52.785610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.786097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.786534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.787246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:52.787701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:52.788127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:52.919370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:52.927080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0e31276-9e49-43c2-94a0-05ee4c2a1399 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152596 Sep 18 06:33:52.927640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 971/1887] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:52 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:53.441106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:53.443290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:53.449532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.450011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.450446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.451521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.451983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.452407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.483694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e2411aad-ca26-4667-89bd-16676ded6ba7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046512 Sep 18 06:33:53.484250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 917/1888] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:53.619625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:53.620535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:53.620770 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:53.622959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:53.627248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.627737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.628206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.628911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.629036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.629367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.629476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.629793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.629898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.630546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.631037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.631495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.644801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:53.661238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ec96892c-c871-4331-814e-ad3f07b297ab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044577 Sep 18 06:33:53.661818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 918/1889] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:53 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:53.673595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:53.675794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:53.681614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.682097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.682536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.683259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.683712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.684137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.713072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ce645233-648d-49e9-a02b-bb815ae192ff tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043822 Sep 18 06:33:53.713845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 919/1890] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:53 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:53.760126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:53.768775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:53.770236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:53.771440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8537d40c-245b-409b-80cb-68df86b254b0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156123 Sep 18 06:33:53.772002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 972/1891] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:53 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:33:53.950495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:53.951648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:53.957534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.958072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.958461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:53.959175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:53.959634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:53.960060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.088958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:54.097309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c586c011-d8eb-4b52-bfa8-dec975dadfa0 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150625 Sep 18 06:33:54.097872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 920/1892] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:53 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:54.495903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:54.498555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:54.504551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:54.505037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:54.505545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.506245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:54.506738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:54.507167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.536107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1c317c48-4d34-4ab2-aa67-8a814d4f59c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044468 Sep 18 06:33:54.536653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 973/1893] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:54.674087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:54.676274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:54.682694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:54.683272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:54.683710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.684402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:54.684855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:54.685282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.714096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-01d7e1d5-d1ce-4cd1-9e00-0f062def37f3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044500 Sep 18 06:33:54.714731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 921/1894] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:54 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:54.725320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:54.727498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:54.734368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:54.734886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:54.735325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.736009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:54.736458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:54.736880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.766535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-06d58bb7-77be-42d6-b84f-a9913d10cd73 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045217 Sep 18 06:33:54.767106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 974/1895] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:54 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:54.797646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:54.798255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:54.804149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:54.804630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:54.805066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.805748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:54.806231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:54.806678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:54.930843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:54.939994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:54.941469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:54.942649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-78c02826-3d39-4baf-981b-0ac92994bd27 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149420 Sep 18 06:33:54.943207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 922/1896] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:54 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:55.122713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:55.123250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:55.129171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.129651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.130090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.130801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.131257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.131680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.261081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:55.268700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6fbac957-ac56-4a38-a9a4-f3f2b7ffe9d2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150160 Sep 18 06:33:55.269293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 975/1897] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:55 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:55.548413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:55.550651 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:55.556660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.557146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.557582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.558270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.558759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.559192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.587974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b471951e-f003-4941-b67a-e4a915e5c2f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043555 Sep 18 06:33:55.588515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 923/1898] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:55.727005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:55.729077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:55.735499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.735980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.736415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.737094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.737544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.737965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.768061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-321a55b9-8a0c-4b5f-96c2-5983a3aaccab tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045229 Sep 18 06:33:55.768608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 976/1899] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:55 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:55.778757 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:55.781644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:55.787921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.788398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.788829 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.789505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.789982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.790408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.819327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ba435aeb-9fa5-4420-8d10-5b29d2129c41 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044505 Sep 18 06:33:55.819865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 924/1900] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:55 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:55.968924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:55.969556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:55.975487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.975971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.976412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:55.977153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:55.977555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:55.977984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.104451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:56.113071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:56.114512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:56.115697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-430d03c4-718c-46dd-9497-a21d6bf1c533 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152117 Sep 18 06:33:56.116258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 977/1901] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:55 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:56.292190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:56.292762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:56.299187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:56.299663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:56.300095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.300771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:56.301215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:56.301634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.428141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:56.436483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-83e4a549-891c-4036-8c1f-f6dbb4ab3a84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148210 Sep 18 06:33:56.437038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 925/1902] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:56 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:56.601189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:56.603321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:56.611880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:56.612362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:56.612801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.613485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:56.613939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:56.614366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.643136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bf5f3dfa-4cbd-435d-83ba-afe6589f8c8f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046053 Sep 18 06:33:56.643701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 978/1903] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:56 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:56.781025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:56.783232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:56.789614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:56.790093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:56.790529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.791234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:56.791693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:56.792118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.821519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-185e10fa-3dcc-4a69-8f2f-821481de7912 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045014 Sep 18 06:33:56.822056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 926/1904] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:56 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:56.831407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:56.833558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:56.839918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:56.840402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:56.840840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.842017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:56.842466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:56.842922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:56.872497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cd3b9227-4217-47ec-9b00-ce9c8f2108de tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044972 Sep 18 06:33:56.872976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 979/1905] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:56 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:57.141933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:57.142462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:57.148489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.148968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.149400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.150082 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.150534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.150982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.280644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:57.289152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:57.290609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:57.291759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-84188cb5-7b2a-4cd7-ae51-68bdb1761bdd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154340 Sep 18 06:33:57.292309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 927/1906] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:57 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:57.458800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:57.459417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:57.465503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.465985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.466419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.467504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.468067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.468394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.597890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:57.605569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-49c89257-b7fd-455e-a490-33f1a9c437e7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150655 Sep 18 06:33:57.606135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 980/1907] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:57 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:57.655336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:57.657556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:57.663431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.663919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.664349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.665030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.665482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.665904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.669469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:57.696599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-52a0f4a2-61fa-4af7-8a19-85f87779beb5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045171 Sep 18 06:33:57.697141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 928/1908] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:57.833935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:57.836020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:57.842408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.842924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.843365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.844047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.844497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.844922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.874941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-55eaa279-14ab-4227-85d6-00f7200474eb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045052 Sep 18 06:33:57.875489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 981/1909] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:57 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:57.884645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:57.886880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:57.893256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.893740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.894176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.894903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:57.895355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:57.895783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:57.924885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9452a057-30ee-4896-b349-00163ac7fdbd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044205 Sep 18 06:33:57.925430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 929/1910] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:57 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:58.316020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:58.316621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:58.322618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.323096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.323529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.324204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.324653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.325072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.454515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:58.463266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:58.464713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:58.465880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b368ac37-c95f-4850-9007-cbb46f5a5c8e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153785 Sep 18 06:33:58.467055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 982/1911] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:58 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:58.629327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:58.629874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:58.635931 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.636409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.636841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.637520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.637964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.638920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.708918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:58.711293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:58.718104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.718604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.719084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.719781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.720241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.720668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.750599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aadb3fa0-679d-4e02-9939-b84c42c10a33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045601 Sep 18 06:33:58.751167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 983/1912] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:58 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:58.766838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:58.774459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b1a8c3d5-ea14-49a4-a372-0848a11add9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149161 Sep 18 06:33:58.775040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 930/1913] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:58 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:33:58.887436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:58.890134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:58.896556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.897031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.897462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.898144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.898613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.899040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.927001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc17d111-189a-4fc2-a9af-9dbefb97d391 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043645 Sep 18 06:33:58.927551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 984/1914] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:58 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:58.937122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:58.939342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:58.945583 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.946061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.946490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.947194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:58.947641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:58.948060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:58.977344 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-277c1e2b-a6f7-4457-b817-95a2ac8def7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044614 Sep 18 06:33:58.977903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 931/1915] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:58 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:59.491259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:59.491867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:59.498157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.498678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.499114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.499797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.500249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.500671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.626618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:59.635076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:59.636511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:59.637705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-48fe8e30-f371-4e55-8332-829c99dc7fda tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150405 Sep 18 06:33:59.639104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 985/1916] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:33:59 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:33:59.763870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:59.766055 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:59.772062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.772548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.772988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.773678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.774136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.774566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.798961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:59.799575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:59.803674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2379ca01-b608-40a4-81ff-6ee73110d51a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044351 Sep 18 06:33:59.804431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 932/1917] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:59 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:33:59.805589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.806074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.806509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.807237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.807690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.808113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.939924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:59.941358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:33:59.942212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:59.944494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:33:59.949544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9e82258a-8e14-4a82-a205-b8dd918bf46e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154344 Sep 18 06:33:59.949712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.950118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 986/1918] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:33:59 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:33:59.950248 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.955153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.955881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.956348 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.956777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.986049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-456f778b-4919-413f-9aeb-a9c96e360b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050581 Sep 18 06:33:59.986788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 933/1919] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:59 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:33:59.988971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:33:59.991165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:33:59.997256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.998348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:33:59.998808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:33:59.999486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:33:59.999931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:00.000351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:00.029751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4f94a4bc-08fe-4918-8513-91703fb013bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044649 Sep 18 06:34:00.030304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 987/1920] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:33:59 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:00.663453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:00.664064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:00.670655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:00.671147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:00.671593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:00.672284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:00.672740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:00.673449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:00.798525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:00.807077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:00.808506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:00.809655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-482a8e0a-efd0-4637-8599-2c01e4dd9599 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150244 Sep 18 06:34:00.810825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 934/1921] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:00 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:00.816613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:00.818813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:00.824786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:00.825272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:00.825714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:00.826396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:00.826880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:00.827308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:00.856636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-034cf23a-1e6a-4925-8b5f-ca653e65d6a6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044536 Sep 18 06:34:00.857680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 988/1922] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:00.972217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:00.972848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:00.979209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:00.979680 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:00.980109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:00.980816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:00.981264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:00.981682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:00.999325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:01.001448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:01.007656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:01.008135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:01.008565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:01.009251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:01.009697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:01.010116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:01.013716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:01.040048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-61c69ca6-8adf-42e6-ba2a-115aca696b2e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045508 Sep 18 06:34:01.043282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 989/1923] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:00 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:01.048972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:01.051142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:01.056972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:01.057454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:01.057889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:01.058594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:01.059050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:01.059473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:01.088717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-046dcf53-ce65-4562-a3f7-2c20b0f127a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044090 Sep 18 06:34:01.089281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 990/1924] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:01 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:01.117442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:01.125475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c79fb867-115c-45e5-8a2f-db085766165a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.157128 Sep 18 06:34:01.126034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 935/1925] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:00 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:01.834973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:01.835499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:01.841488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:01.841967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:01.842410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:01.843120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:01.843572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:01.843993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:01.869464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:01.871938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:01.878422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:01.878940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:01.879373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:01.880064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:01.880516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:01.880936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:01.910674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-23e58df8-2eff-4984-8e75-8af09ff39f4a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045200 Sep 18 06:34:01.911237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 936/1926] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:01.969896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:01.978390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:01.979874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:01.981040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-763f0ecd-3043-4b8d-ab61-35dbd0839fb8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150113 Sep 18 06:34:01.981600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 991/1927] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:01 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:02.052954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:02.055138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:02.061967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:02.062444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:02.062904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:02.063597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:02.064033 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:02.064452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:02.093774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6c926688-c697-4c61-af53-7fbce52e6b1e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044795 Sep 18 06:34:02.094235 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 937/1928] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:02 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:02.102131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:02.104301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:02.110849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:02.111330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:02.111762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:02.112443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:02.112888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:02.113307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:02.143195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8836f9d2-edb1-44d7-90ab-805882f1145b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045878 Sep 18 06:34:02.143747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 992/1929] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:02 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:02.149170 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:02.149785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:02.155639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:02.156112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:02.156539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:02.157214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:02.157664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:02.158084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:02.286991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:02.295187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6ae41b49-1555-49fb-91a1-ec0f43989554 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149358 Sep 18 06:34:02.295737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 938/1930] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:02 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:02.923880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:02.926067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:02.932645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:02.933132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:02.933567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:02.934258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:02.934750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:02.935204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:02.964404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-980452d5-3c6c-4fc2-908e-e71898187c24 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045156 Sep 18 06:34:02.964958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 993/1931] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:03.006559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:03.007195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:03.013637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.014121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.014561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.015274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.015731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.016163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.106336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:03.108553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:03.114940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.115429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.115866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.116565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.117019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.117481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.142799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:03.147541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1527a865-760d-4794-8072-f191a1ac28a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045059 Sep 18 06:34:03.148118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 994/1932] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:03 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:03.151897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:03.153359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:03.155148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cd3acafa-d728-4f08-abd5-603eb1cf0594 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.152496 Sep 18 06:34:03.155698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 939/1933] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:03 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:03.156393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:03.158527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:03.164288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.164768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.165206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.165899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.166353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.166813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.195361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-86a3fc80-6e2e-4d82-9d19-ba5710437a20 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043908 Sep 18 06:34:03.195846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 995/1934] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:03 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:03.318741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:03.319364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:03.325268 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.326323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.326812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.327508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.327967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.328397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.455715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:03.463705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4e624534-7fc6-4615-ab2c-9fe32a07d122 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148753 Sep 18 06:34:03.464278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 940/1935] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:03 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:03.976939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:03.979123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:03.985716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.986194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.986651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:03.987337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:03.987790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:03.988215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.017763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8f104b43-ab04-4393-8741-e6e9c3acb108 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044773 Sep 18 06:34:04.018316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 996/1936] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:04.159701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:04.161848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:04.168363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:04.168844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:04.169954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.170657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:04.171112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:04.171534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.179585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:04.180199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:04.186617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:04.187228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:04.187671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.188362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:04.188813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:04.189236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.200692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9980fc21-d8c1-4f2f-8a06-045df357daf9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044933 Sep 18 06:34:04.202390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 941/1937] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:04 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:04.207368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:04.209502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:04.215213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:04.215693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:04.216672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.217392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:04.217838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:04.218255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.248397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-714cdf5e-1f25-4eea-a243-03d4c56fa5fe tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044600 Sep 18 06:34:04.248866 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 942/1938] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:04 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:04.320590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:04.329088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:04.330516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:04.331702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0267b5cc-1f81-44b6-9760-ad52cf0ca1b7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156027 Sep 18 06:34:04.332259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 997/1939] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:04 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:34:04.487333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:04.488048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:04.494077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:04.494561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:04.495282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.496023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:04.496476 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:04.496918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:04.624366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:04.632113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-180d3640-9733-4ba6-977f-7ee9e74102e1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148773 Sep 18 06:34:04.632679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 943/1940] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:04 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:05.030949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:05.033118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:05.039121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.039605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.040042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.040730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.041182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.041615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.071239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6c44df1c-0707-4999-a02a-38e7c02c0ded tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045095 Sep 18 06:34:05.071784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 998/1941] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:05.213175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:05.215380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:05.221822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.222303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.222797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.223504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.223956 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.224379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.249606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:05.254835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-64738f6a-8cb1-417a-8c9b-e8f4757ccfb5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045675 Sep 18 06:34:05.255379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 944/1942] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:05 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:05.261100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:05.263348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:05.269747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.270224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.270682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.271366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.271814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.272238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.301249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-24ef4401-54a4-41b9-acfe-b20a71536b6c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044040 Sep 18 06:34:05.301728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 999/1943] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:05 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:05.357596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:05.358236 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:05.364126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.364610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.365049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.365736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.366187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.366630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.498902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:05.507706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:05.509172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:05.510329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9f3eaf4b-5953-4a59-871e-4a0a31d5aecf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157336 Sep 18 06:34:05.510911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 945/1944] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:05 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:05.656177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:05.656783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:05.662726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.663201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.663633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.664308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:05.664755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:05.665172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:05.793711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:05.802038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f143a9db-5894-4a0f-955f-49f9d87ffc00 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150185 Sep 18 06:34:05.802624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1000/1945] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:05 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:06.084295 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:06.086349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:06.092845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.093328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.093761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.094446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.094936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.095359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.123937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0bdf3372-4b7d-42eb-a930-be3a0ed434cc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043725 Sep 18 06:34:06.124470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 946/1946] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:06.266833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:06.269046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:06.275487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.275967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.276404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.277089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.277540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.277963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.306880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6866d9c2-a206-4434-b9d9-eb02eed5d782 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044578 Sep 18 06:34:06.307635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1001/1947] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:06 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:06.313936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:06.316112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:06.322720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.323212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.323642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.324326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.324778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.325217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.354804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f86c344d-82c1-46be-bcdb-5a9534459caa tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045292 Sep 18 06:34:06.355277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 947/1948] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:06 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:06.535236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:06.535848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:06.537675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:06.543117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.543322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.544384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.544530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.544946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.545372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.670943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:06.679391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:06.680832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:06.681986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b28d6fae-adf0-4bf1-9927-28d915a3c89d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150697 Sep 18 06:34:06.682542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1002/1949] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:06 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:06.825099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:06.825715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:06.831769 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.832255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.832693 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.833382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:06.833837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:06.834270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:06.964263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:06.971901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-efc05371-5bab-4364-87f2-cf30c9113fa7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150695 Sep 18 06:34:06.972468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 948/1950] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:06 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:07.136347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:07.138479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:07.144444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:07.144916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:07.145347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:07.146027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:07.146473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:07.146916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:07.181837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bceb1c05-fea1-4a80-86a5-0a041643b09e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049319 Sep 18 06:34:07.182323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1003/1951] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:07.319826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:07.321997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:07.328540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:07.329025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:07.329468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:07.330161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:07.330681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:07.331110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:07.361087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a3d0edd1-64c4-4a6e-9527-2fba788e5a38 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045328 Sep 18 06:34:07.361817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 949/1952] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:07 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:07.367201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:07.369337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:07.375923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:07.376408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:07.376841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:07.377521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:07.377970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:07.378422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:07.407997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f2b86584-3022-4693-b481-c601857e17f0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044686 Sep 18 06:34:07.408567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1004/1953] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:07 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:07.708416 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:07.709028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:07.714970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:07.715400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:07.716468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:07.717188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:07.717642 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:07.718123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:07.854263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:07.863072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:07.864543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:07.865984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c8002f2c-c3c5-4d8b-8a71-8e3dc1186ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.161160 Sep 18 06:34:07.866553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 950/1954] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:07 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:07.995659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:07.996190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:08.002647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.003127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.003557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.004231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.004678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.005099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.134065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:08.136260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:08.143619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f0c11506-99a3-442c-b5b6-d94c38662809 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152095 Sep 18 06:34:08.144201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1005/1955] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:07 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:08.194605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:08.196758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:08.202885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.203371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.203808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.204504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.204958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.205380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.234478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f05cd98f-ad4a-42b6-b75b-fcbfba337155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043912 Sep 18 06:34:08.235059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 951/1956] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:08.374157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:08.376237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:08.382782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.383260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.383696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.384374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.384822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.385241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.413391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9201384d-23fc-499d-ac84-540886e84ddc tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044093 Sep 18 06:34:08.413946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1006/1957] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:08 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:08.420492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:08.422739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:08.429015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.429494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.429929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.430641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.431091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.431512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.463879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ccdb6bd1-bfaa-434e-af15-1dd895b68bf5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047363 Sep 18 06:34:08.464787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 952/1958] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:08 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:08.891944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:08.892547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:08.898466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.899023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.899523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:08.900224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:08.900671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:08.901091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.026497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:09.035510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:09.036942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:09.038125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1edb5a94-c5be-46df-9b5b-6e817a4527df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150117 Sep 18 06:34:09.038694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1007/1959] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:09.166893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:09.167513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:09.173440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:09.173923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:09.174361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.175071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:09.175527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:09.175976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.247410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:09.249554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:09.255645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:09.256132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:09.256570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.257265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:09.257723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:09.258150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.289235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-450c4f3b-41d7-4fb5-b53d-8d8fe316835f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046030 Sep 18 06:34:09.289799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1008/1960] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:09.308090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:09.316166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2e9f5807-91cd-41af-8d5f-0b13e495caf1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153124 Sep 18 06:34:09.316718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 953/1961] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:09 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:09.425515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:09.428078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:09.434985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:09.435474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:09.435909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.436599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:09.437054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:09.437486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.467659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4dc0ac22-f591-41d2-9d51-c2fef1454dd0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046172 Sep 18 06:34:09.468230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1009/1962] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:09 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:09.476550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:09.478755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:09.485001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:09.485478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:09.485912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.486631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:09.487084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:09.487521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:09.517269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-98567fb1-f137-474c-98fa-c30cf4d4bbb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044643 Sep 18 06:34:09.517811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 954/1963] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:09 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:10.062231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:10.062888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:10.068894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.069368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.069801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.070476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.070946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.071359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.197819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:10.206442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:10.207919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:10.209105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-721d5076-9833-4dd2-a992-fafa84c2f6e7 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150800 Sep 18 06:34:10.209656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1010/1964] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:34:10.301569 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:10.303725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:10.309629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.310111 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.310549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.311259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.311713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.312142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.339750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:10.340296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:10.341368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-20c5b2ec-83b5-418b-936c-e0b9595c68e1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043188 Sep 18 06:34:10.341925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 955/1965] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:10.346909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.347395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.347833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.348528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.348985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.349413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.480220 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:10.482407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:10.483134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:10.488822 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.489333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.489757 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.490459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.490949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.491170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-475dc550-452b-46d9-9bf3-03b07a14a90b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155408 Sep 18 06:34:10.491381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.491748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1011/1966] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:10 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:34:10.520753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dd06bd4c-c446-4676-be61-5cd001fa6c08 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044509 Sep 18 06:34:10.521308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 956/1967] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:10 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:10.529770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:10.531970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:10.537789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.538275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.538751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.539440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:10.539897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:10.540327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:10.569541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-09f85f30-6a1f-4fa1-aec4-afd282a398d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044271 Sep 18 06:34:10.570099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1012/1968] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:10 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:11.234856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:11.235496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:11.241485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.241970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.242434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.243141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.243593 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.244021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.353522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:11.356786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:11.362971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.363460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.363898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.364589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.365042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.365466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.375384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:11.383857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:11.385287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:11.386440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-81f998ad-86b5-45eb-91ff-01e6b503870a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156017 Sep 18 06:34:11.387116 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 957/1969] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:11 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:11.395499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0a42863f-679b-414f-a381-1a7415377b9b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045918 Sep 18 06:34:11.396053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1013/1970] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:11.514340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:11.514900 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:11.521285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.521764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.522197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.522912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.523364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.523781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.533683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:11.535877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:11.542197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.542704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.543144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.544408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.544857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.545276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.577890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5ac0ad64-7705-40a4-8b2c-2cd705dba6c5 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048008 Sep 18 06:34:11.578465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1014/1971] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:11.582962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:11.585090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:11.591384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.591874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.592313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.593106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:11.593558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:11.593984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:11.624220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cf80f74a-1cd9-4cef-b205-c6287d6206cf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044659 Sep 18 06:34:11.624800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1015/1972] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:11 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:11.657609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:11.665371 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d51a6acc-69e2-484d-b8c3-1aa1ccea2c69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155001 Sep 18 06:34:11.665935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 958/1973] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:11 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:12.408012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:12.410301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:12.411091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:12.411707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:12.416478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.416963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.417405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.417583 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.418096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.418198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.418536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.418671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.418996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.419247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.419694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.420112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.450626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e64c06bf-c95d-499b-b326-fbcc00ba1b83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045814 Sep 18 06:34:12.451233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1016/1974] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:12.555995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:12.565004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:12.566433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:12.567615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7711d60c-782e-4bcd-835e-251fd0e62465 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.160305 Sep 18 06:34:12.568159 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 959/1975] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:12 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:34:12.590632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:12.592771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:12.599084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.599563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.599993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.600670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.601116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.601536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.630668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b81e560c-a5ab-41ec-ba8f-9f4eeca8b148 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043951 Sep 18 06:34:12.631398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1017/1976] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:12 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:12.636263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:12.639088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:12.644890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.645363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.645794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.646474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.646949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.647368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.676191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-051d4ded-5e43-4787-92f3-ea619d31d2d6 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043673 Sep 18 06:34:12.676665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 960/1977] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:12 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:12.689390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:12.689990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:12.695789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.696265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.696692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.697370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:12.697819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:12.698233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:12.826762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:12.834914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dc7c5b5f-0580-432d-856f-589421938860 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148865 Sep 18 06:34:12.835323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1018/1978] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:12 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:13.462918 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:13.465265 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:13.471880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.472365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.472803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.473490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.473942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.474367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.490563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:13.504099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b4086f16-e5c0-42d4-ad83-0f8dbf1532aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045199 Sep 18 06:34:13.504656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 961/1979] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:13 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:13.593911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:13.594542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:13.601073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.601584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.602015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.602724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.603174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.603589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.642954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:13.645262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:13.651758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.652319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.652830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.654240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.654804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.655232 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.685439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0a610b17-a1eb-434c-9bcc-c61847794fae tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046377 Sep 18 06:34:13.686010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 962/1980] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:13 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:13.690519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:13.692679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:13.698377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.698888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.699324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.700514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.700967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.701390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.731006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-73cf567b-b7af-414a-b711-c2f1a9b96a5b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043807 Sep 18 06:34:13.731462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 963/1981] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:13 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:13.735833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:13.742474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:13.743935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:13.745363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-da76f446-740d-43aa-a1c7-1da83e45dbd6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156000 Sep 18 06:34:13.745919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1019/1982] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:13 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:13.857139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:13.857752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:13.863943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.864457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.864962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.865852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:13.866303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:13.866759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:13.997365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:14.005012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e3fb8583-e654-40a8-8efe-8c501044e42b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151691 Sep 18 06:34:14.005574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 964/1983] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:13 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:14.516463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:14.518802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:14.524791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:14.525267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:14.525750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:14.526381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:14.526884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:14.527324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:14.557073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-37409663-7795-4b96-91db-fb7cc362227d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045239 Sep 18 06:34:14.557620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1020/1984] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:14.697703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:14.700448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:14.706998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:14.707513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:14.707950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:14.708634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:14.709088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:14.709513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:14.737665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c0937aee-54a1-4a4d-ae69-886780fc05a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043937 Sep 18 06:34:14.738373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 965/1985] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:14.743547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:14.746047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:14.753691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:14.754168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:14.754630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:14.755314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:14.755761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:14.756179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:14.770677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:14.771320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:14.777200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:14.777644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:14.778118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:14.778937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:14.779390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:14.779816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:14.785969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-708f9fe8-153d-4ed6-86f4-fc69e2e7ddc1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046284 Sep 18 06:34:14.786530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1021/1986] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:14 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:14.905143 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:14.913722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:14.915185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:14.916378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3e2d33b3-c6ab-40f1-9772-dbf27ba63858 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149686 Sep 18 06:34:14.916933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 966/1987] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:14 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:34:15.028976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:15.029514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:15.035391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.035863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.036290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.036966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.037413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.037829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.052537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:15.245302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:15.252849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ec1eae2f-19ae-41f1-b8a4-bb2d876e6e6b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.228471 Sep 18 06:34:15.253424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1022/1988] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:15 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:15.569401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:15.571588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:15.577510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.577996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.578433 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.579153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.579610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.580035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.609887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dc6f223b-f9d0-4170-a6f1-221f2f75ecc9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044478 Sep 18 06:34:15.610448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 967/1989] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:15.749462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:15.751562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:15.757953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.758458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.758939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.759627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.760086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.760515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.790130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7ac58f35-d731-41ca-baab-aa8223c38a6a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044776 Sep 18 06:34:15.790709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1023/1990] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:15 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:15.798283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:15.800435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:15.807002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.807485 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.807999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.808682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.809155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.809576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.844423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7de7eacd-f666-428e-abe4-ee64d0be064a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049514 Sep 18 06:34:15.845345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 968/1991] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:15 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:15.941207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:15.941835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:15.947893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.948380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.948826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:15.949517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:15.950495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:15.950952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.077134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:16.085733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:16.087241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:16.088414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-44df016a-3397-48f7-b573-e893f406d05b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.151160 Sep 18 06:34:16.088995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1024/1992] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:15 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:16.276058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:16.276753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:16.283146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:16.283632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:16.284068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.284756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:16.285208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:16.285635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.460818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:16.469171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f01f3b14-c5dc-4c9e-8bd8-4e83e710daaf tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.196949 Sep 18 06:34:16.469772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 969/1993] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:16 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:16.623039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:16.625174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:16.631188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:16.631665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:16.632096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.632774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:16.633220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:16.633640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.663076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b5819455-1aff-4610-a332-d1f518afbf6f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044299 Sep 18 06:34:16.663615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1025/1994] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:16.802053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:16.804237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:16.811141 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:16.811618 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:16.812050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.812729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:16.813178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:16.813596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.843325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-410822c8-fa1a-4c65-b53a-8c2e480e1dd7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045267 Sep 18 06:34:16.843872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 970/1995] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:16.857000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:16.859161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:16.865466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:16.865947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:16.866381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.867110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:16.867562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:16.867989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:16.901074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc1c7353-db20-42e8-a01a-7f4503751759 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046778 Sep 18 06:34:16.901654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1026/1996] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:16 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:17.114436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:17.114994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:17.121193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.122161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.122609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.123292 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.123741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.124161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.256721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:17.265326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:17.266790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:17.267948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-89fe5832-d745-4c64-ad44-97c617eb1ab6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157476 Sep 18 06:34:17.268501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 971/1997] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:17 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:17.492078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:17.492723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:17.499707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.500195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.500635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.501327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.501786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.502215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.631049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:17.639053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-75ca0728-1adf-44b6-a6f6-a7e1379947ac tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150940 Sep 18 06:34:17.639619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1027/1998] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:17 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:17.675521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:17.677671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:17.683632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.684117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.684554 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.685242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.685728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.686158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.714313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f55e6943-65f9-4ad5-866e-1ae8cd029f64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042735 Sep 18 06:34:17.714945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 972/1999] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:17.855645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:17.858448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:17.864553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.865032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.865471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.866196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.866788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.867218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.897147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fa995bc4-3c35-48ed-b54a-77ebfce397fd tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045397 Sep 18 06:34:17.897705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1028/2000] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:17 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:17.915415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:17.917542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:17.924432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.924915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.925351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.926031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:17.926487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:17.926938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:17.956533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7dec7258-a49b-4c17-a0ff-91186ad6c492 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045099 Sep 18 06:34:17.957075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 973/2001] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:17 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:18.292709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:18.293318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:18.299355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.299836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.300273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.300956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.301410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.301843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.430324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:18.439610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:18.441054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:18.442249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eb4dfc9d-8017-4eb5-b21d-7ad728e7c054 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.153420 Sep 18 06:34:18.442837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1029/2002] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:18 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:18.662522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:18.663164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:18.669929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.670410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.670868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.671548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.671997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.672443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.726310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:18.728635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:18.734555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.735077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.735516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.736209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.736682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.737112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.767714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-88e00fb3-1a7b-41d5-89af-9269989b3c45 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045363 Sep 18 06:34:18.768282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1030/2003] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:18.804498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:18.812191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5bd865fc-edc3-4777-9c32-f3e357ee2b58 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153512 Sep 18 06:34:18.812747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 974/2004] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:18 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:18.909976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:18.912183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:18.918141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.918653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.919091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.919775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.920225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.920649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.949999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1eeebf38-0027-4738-9f29-9f3a0d9b5955 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043655 Sep 18 06:34:18.950543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1031/2005] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:18 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:18.969247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:18.971422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:18.977274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.977753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.978187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:18.978897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:18.979346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:18.979766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.008326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e00a3361-1bc4-4076-93c3-16592d8fbd81 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043629 Sep 18 06:34:19.008857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 975/2006] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:18 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:19.467614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:19.468223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:19.474274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:19.474798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:19.475235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.475920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:19.476371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:19.476797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.611770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:19.620716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:19.622871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:19.624402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-61e4aa95-4391-4146-8f0c-81057bbf5f0f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.160410 Sep 18 06:34:19.624995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1032/2007] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:19 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:19.780815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:19.782880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:19.790635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:19.791115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:19.791543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.792227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:19.792707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:19.793213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.823641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b2d6b12c-6b0d-4c9a-a3a7-a73dc7414438 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047252 Sep 18 06:34:19.824113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 976/2008] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:19.835397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:19.836010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:19.842566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:19.843072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:19.843505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.844183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:19.844629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:19.845044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.961448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:19.963652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:19.969928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:19.970553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:19.970879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.971573 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:19.972021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:19.972442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:19.975197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:19.983682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34da5f0f-5e0e-4063-94df-3042ddc8ae96 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152196 Sep 18 06:34:19.984259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1033/2009] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:19 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:20.003136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-86e735bc-f92f-449a-806c-4891bc5b2ac3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045592 Sep 18 06:34:20.003692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 977/2010] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:19 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:20.020865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:20.023072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:20.029551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:20.030030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:20.030463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:20.031242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:20.031637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:20.032062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:20.062440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5bb207e7-0b91-42ba-8c40-52b01b919d9f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045487 Sep 18 06:34:20.063029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1034/2011] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:20 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:20.649701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:20.650307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:20.656437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:20.656916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:20.657349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:20.658028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:20.658474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:20.658916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:20.662801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:20.783731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:20.792640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:20.794070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:20.795254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-70cb2267-c074-458f-b923-6301e450cf0a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.149483 Sep 18 06:34:20.795810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 978/2012] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:20 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:20.896876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:20.899543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:20.904980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:20.905452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:20.905883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:20.906558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:20.907025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:20.907443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:20.936776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-72653011-7a68-4c19-b3da-7ce61b84be17 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.104767 Sep 18 06:34:20.937327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1035/2013] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:20 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:21.007540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:21.008095 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:21.014018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.014511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.014990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.015114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:21.015684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.016139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.016566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.017168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:21.022893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.023369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.023805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.024485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.024934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.025354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.054137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c22ac56a-3cc0-4345-b4e9-aa28c2b508e7 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042795 Sep 18 06:34:21.054726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1036/2014] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:21 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:21.074716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:21.076844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:21.082745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.083228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.083662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.084363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.084813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.085233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.114746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b0cbb1af-9164-48a1-878a-121269bc2907 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043815 Sep 18 06:34:21.115517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1037/2015] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:21 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:21.149042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:21.157368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b1fd849c-ea49-4b71-b20e-f106fbbea89e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153743 Sep 18 06:34:21.157930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 979/2016] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:21 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:21.821156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:21.821761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:21.828286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.828761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.829201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.829882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.830335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.830783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.950147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:21.952367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:21.958805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.959296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.959736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.960439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:21.960897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:21.961333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:21.961437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:21.970643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:21.972128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:21.973347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3084d94a-8f7f-4a5a-aadb-f867e2aa671f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.156169 Sep 18 06:34:21.973921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1038/2017] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:21 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:22.067802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:22.070122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:22.077368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:22.077853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:22.078296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:22.079025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:22.079476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:22.079900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:22.082709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:22.112257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cc5bca6d-cd83-4967-aa0c-3f1485b59c00 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048657 Sep 18 06:34:22.112832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1039/2018] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:22 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:22.127220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:22.129339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:22.135084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:22.135561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:22.135991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:22.136667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:22.137665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:22.138086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:22.140942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-588593a7-bb60-4592-b80b-e5982e42df97 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.195412 Sep 18 06:34:22.145783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 980/2019] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:22.166891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e64ab021-658d-4e59-969a-8e802ee82627 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043385 Sep 18 06:34:22.167452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1040/2020] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:22 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:22.179933 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:22.180549 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:22.186992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:22.187477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:22.187915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:22.188603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:22.189061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:22.189490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:22.337152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:22.345258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dff99be9-2e6a-4013-a456-d154927b8726 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.169020 Sep 18 06:34:22.345892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 981/2021] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:22 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:22.999952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:23.000561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:23.007015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.007492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.007929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.008618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.009066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.009488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.125112 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:23.127289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:23.133667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.134148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.134599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.135285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:23.135750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.135750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.136139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.143974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:23.145420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:23.146635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d55c6adc-7f2e-49ee-bea8-1c691982d2b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.150577 Sep 18 06:34:23.147420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1041/2022] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:22 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:23.153999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:23.156321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:23.162388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.163021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.163557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.164917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.165374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.165798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.165989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-33f57dcc-941d-434f-a248-2068860ac0f9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044831 Sep 18 06:34:23.166543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 982/2023] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:23 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:23.178327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:23.180447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:23.182219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:23.187347 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.187828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.188258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.188942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.189387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.189804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.197501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8cb6e71e-e9c2-4dc2-822a-f936ec635c5b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047994 Sep 18 06:34:23.198067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1042/2024] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:23.219869 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-eb64752d-7775-4ec4-89e1-d004a3158b28 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045190 Sep 18 06:34:23.220409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 983/2025] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:23 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:23.368273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:23.368799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:23.375005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.375486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.375922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.376604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:23.377055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:23.377480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:23.507653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:23.515821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e1c48bc6-89aa-4c02-bb24-b15c685eff84 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151496 Sep 18 06:34:23.516394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1043/2026] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:23 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:24.172311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:24.172920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:24.178873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:24.179505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.179997 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.180436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.181045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:24.181161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.181564 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.181992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.187856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.188327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.188755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.189433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.189877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.190293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.220251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5793470d-217b-40d2-a0f6-d10518700504 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045248 Sep 18 06:34:24.220814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1044/2027] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:24 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:24.225690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:24.227943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:24.233831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.234312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.234780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.235462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.235908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.236330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.266544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1a9ef94f-96f9-49e6-897b-f92b1737eae3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044523 Sep 18 06:34:24.267138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1045/2028] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:24 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:24.271666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:34:24.273796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:24.280200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.280684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.281126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.281813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.282262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.282713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.309597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR oslo.messaging._drivers.impl_rabbit [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [625d2a70-a6de-4e6b-8ecd-ff4a9a18d7eb] AMQP server on 10.222.0.20:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 18 06:34:24.310932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:24.319828 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:24.321257 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:24.322412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-14fdc8fb-8192-4e1d-9806-96a34377740b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.154063 Sep 18 06:34:24.323072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 984/2029] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:24 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 06:34:24.539354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:24.539967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:24.545878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.546355 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.546811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.547491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:24.547938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:24.548359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:24.674973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:24.682530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1ac3e3fa-f21d-4d0b-8b39-08ff71cc3c9e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147093 Sep 18 06:34:24.683129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 985/2030] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:24 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:34:25.233288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:25.235507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:25.242110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:25.242654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:25.243088 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:25.243771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:25.244496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:25.244901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:25.274463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-35b6573a-0284-4b37-a365-04983e657b32 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045740 Sep 18 06:34:25.275028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 986/2031] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:25 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:25.280011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:25.282034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:25.287613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:25.288090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:25.288522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:25.289200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:25.289648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:25.290067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:25.316983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [625d2a70-a6de-4e6b-8ecd-ff4a9a18d7eb] Reconnected to AMQP server on 10.222.0.20:5672 via [amqp] client with port 47858. Sep 18 06:34:25.320505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-61ca645a-a6e1-4009-b7ad-d946f21e089b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044400 Sep 18 06:34:25.321064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 987/2032] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:25.346886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:25.347496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:25.353269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:25.353753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:25.354192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:25.354908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:25.355369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:25.355795 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:25.378523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-abe4291a-ceb4-48b4-92d4-23c51fea8eb1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/action" status: 200 len: 41088 microversion: 2.1 time: 1.110296 Sep 18 06:34:25.385417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1046/2033] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:34:24 2022] POST /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695/action => generated 41088 bytes in 1112 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:34:25.488598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:25.497830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:25.499357 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:25.500582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-176f64e0-db1e-4d19-9d1f-9ce8d22c9cb3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 200 len: 1752 microversion: 2.20 time: 0.157343 Sep 18 06:34:25.501129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 988/2034] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 06:34:25 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 1752 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 06:34:25.706142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:25.706883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:25.713822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:25.714307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:25.714779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:25.715464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:25.715916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:25.716343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:25.847201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:25.856029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cecb3cbd-dd02-4ef6-a092-96f5ba64bd6c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.153551 Sep 18 06:34:25.856586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1047/2035] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:25 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:26.287415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:26.289540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:26.296514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:26.296998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:26.297448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:26.298176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:26.298654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:26.299083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:26.328463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7f4e9fb3-a172-4dee-93b1-8d7303aaaba0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044999 Sep 18 06:34:26.329020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 989/2036] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:26 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:26.334036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:26.336365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:26.342852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:26.343331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:26.343770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:26.344467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:26.344917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:26.345336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:26.376202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0e5028e8-0a3b-4f4b-ada2-c1366a0c8287 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046786 Sep 18 06:34:26.376747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1048/2037] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:26.879394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:26.879958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:26.885827 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:26.886304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:26.886771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:26.887455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:26.887905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:26.888326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:27.017885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:27.025464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2d270d31-e4eb-47d9-8f6a-04ae4d15fd49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150008 Sep 18 06:34:27.026020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 990/2038] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:27.340022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:27.342270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:27.348715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:27.349194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:27.349629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:27.350314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:27.350794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:27.351223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:27.379641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0ee2d460-545d-47aa-bba7-2237f94f33bb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043615 Sep 18 06:34:27.380187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1049/2039] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:27 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:27.389221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:27.391385 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:27.397709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:27.398191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:27.398655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:27.399343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:27.399799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:27.400223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:27.428786 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-445edaad-dfa5-4ea6-972b-8d0658734cd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044147 Sep 18 06:34:27.429338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 991/2040] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:28.048317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:28.048924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:28.055399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:28.055879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:28.056312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:28.056991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:28.057448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:28.057867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:28.185884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:28.193574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7ca6d36e-c4c8-4bde-a144-7a641452be49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149186 Sep 18 06:34:28.194146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1050/2041] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:28 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:28.392825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:28.394957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:28.401396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:28.401874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:28.402311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:28.403040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:28.403494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:28.403920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:28.432316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ced51069-15b6-40dc-8506-7bd42cd37273 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043547 Sep 18 06:34:28.432856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 992/2042] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:28 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:28.441200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:28.443371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:28.445403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:34:28.445544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:34:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments (ip 10.222.0.20) !!! Sep 18 06:34:28.451204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:28.451683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:28.452115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:28.452799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:28.453253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:28.453665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:28.485579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d4dd94f0-63a0-4d3e-88f7-2e44bf26fd82 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048257 Sep 18 06:34:28.486126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1051/2043] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:28 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:29.217797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:29.218353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:29.224383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:29.224876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:29.225318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:29.226013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:29.226482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:29.226926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:29.353418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:29.361123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-71894545-18e0-4f58-a310-68c38bfc5e85 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147868 Sep 18 06:34:29.361689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 993/2044] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:29 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:29.445611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:29.447807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:29.450689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:29.456010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:29.456493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:29.456932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:29.457622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:29.458071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:29.458494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:29.490959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aa968563-7794-4853-ada4-433e75583685 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049297 Sep 18 06:34:29.491524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1052/2045] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:29 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:29.497772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:29.499979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:29.506289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:29.506809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:29.507240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:29.507929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:29.508375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:29.508794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:29.535249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:29.540985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c9dc2030-e8f4-48ee-a578-240fdcc16155 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047460 Sep 18 06:34:29.541535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 994/2046] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:30.384548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:30.385190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:30.391894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:30.392406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:30.392841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:30.393519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:30.393967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:30.394383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:30.504128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:30.506350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:30.513397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:30.513952 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:30.514392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:30.515105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:30.515553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:30.515977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:30.527366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:30.535173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ad9707c7-b445-47be-9cd1-f3dfcd8ac69b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154485 Sep 18 06:34:30.535750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1053/2047] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:30 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:30.546608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a40decaa-6ddb-40fe-9415-f69bc87a259e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046348 Sep 18 06:34:30.548299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 995/2048] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:30 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:30.553408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:30.555830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:30.562397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:30.562913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:30.563351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:30.564182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:30.564675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:30.565116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:30.594972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6d8b4508-ae01-4778-b2ec-08e7bd717d3d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045568 Sep 18 06:34:30.595530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1054/2049] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:30 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:31.558887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:31.559571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:31.560285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:31.562462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:31.566184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:31.566692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:31.567130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:31.567827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:31.568278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:31.568701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:31.568879 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:31.569331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:31.569761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:31.570445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:31.570920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:31.571342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:31.599763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4f05354e-164f-4a9f-ae37-e9e6f69da803 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043972 Sep 18 06:34:31.600328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 996/2050] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:31 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:31.611954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:31.614094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:31.620234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:31.621397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:31.621836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:31.622529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:31.623010 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:31.623437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:31.659753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ca616fd7-4591-42b7-a6f1-77df7c722290 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051578 Sep 18 06:34:31.660441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 997/2051] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:31.701247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:31.708965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b17f1aed-a31b-484c-afcd-2e4e243a9f04 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154513 Sep 18 06:34:31.709534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1055/2052] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:31 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:32.612701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:32.614885 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:32.621640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:32.622163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:32.622625 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:32.623356 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:32.623835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:32.624306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:32.653302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e9747f74-4b71-4091-a507-26b41bd14298 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044674 Sep 18 06:34:32.653840 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 998/2053] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:32 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:32.672882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:32.675084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:32.682185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:32.682746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:32.683241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:32.683964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:32.684426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:32.684854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:32.688641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:32.716474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c991b8e7-5455-4db5-bfd6-f677a47df789 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047582 Sep 18 06:34:32.716974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1056/2054] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:32 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:32.732329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:32.732943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:32.738984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:32.739451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:32.739887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:32.740574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:32.741025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:32.741449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:32.875065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:32.882775 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-293c506f-da02-4714-b73b-41fc9ffd51a4 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154900 Sep 18 06:34:32.883332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 999/2055] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:32 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:33.665497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:33.668163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:33.674190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:33.674791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:33.675130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:33.675807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:33.676261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:33.676684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:33.706285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2cf0d23f-36aa-4f78-9fb7-668da17cf071 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044756 Sep 18 06:34:33.706879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1057/2056] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:33 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:33.728317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:33.730968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:33.737368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:33.737861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:33.738290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:33.739013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:33.739468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:33.739892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:33.768993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fcefebdd-d7a6-44db-b393-64cdf353b928 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044712 Sep 18 06:34:33.769525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1000/2057] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:33 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:33.905724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:33.906332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:33.912301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:33.912784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:33.913228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:33.913911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:33.914364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:33.914823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:34.041450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:34.049780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-efa2dcb3-7d20-4e48-b8bb-f9161f3c153c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147922 Sep 18 06:34:34.050350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1058/2058] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:33 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:34.719585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:34.721674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:34.727731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:34.728208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:34.728637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:34.729354 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:34.729803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:34.730229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:34.759124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5712dd1d-699e-4e30-9219-1baf7f585553 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044364 Sep 18 06:34:34.759672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1001/2059] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:34 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:34.781983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:34.784168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:34.790711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:34.791192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:34.791629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:34.792315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:34.792888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:34.793314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:34.822195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f0f85095-e259-41a2-9ec4-15b3f8d6ecf4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044232 Sep 18 06:34:34.822753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1059/2060] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:35.072478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:35.073032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:35.079661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:35.080139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:35.080571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:35.081249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:35.081700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:35.082147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:35.210854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:35.218535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2168feeb-bb42-49ac-9c06-69bb42253fed tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150004 Sep 18 06:34:35.219121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1002/2061] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:35 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:35.772583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:35.774827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:35.780802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:35.781278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:35.781704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:35.782379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:35.782983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:35.783253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:35.811869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ba62204a-b422-4669-90b4-fc06029529a0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043652 Sep 18 06:34:35.812421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1060/2062] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:35 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:35.835129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:35.837262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:35.843765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:35.844245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:35.844677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:35.845364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:35.845816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:35.846238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:35.875369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f11a7514-2f58-4e47-af7b-47aebd39eaf8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044194 Sep 18 06:34:35.875902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1003/2063] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:36.242556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:36.243189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:36.250139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:36.250690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:36.251129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:36.251815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:36.252267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:36.252691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:36.380145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:36.387869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f9d11e73-7ff8-487d-acad-3b4218744b94 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149392 Sep 18 06:34:36.388440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1061/2064] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:36 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:36.824939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:36.827133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:36.833109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:36.833592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:36.834023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:36.834737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:36.835189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:36.835744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:36.865776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-00c27abc-1e01-4675-87be-12656325cede tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045254 Sep 18 06:34:36.866349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1004/2065] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:36 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:36.887857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:36.890102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:36.896624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:36.897137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:36.897581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:36.898270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:36.898756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:36.899182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:36.929473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-72150602-6cad-4bcd-be07-29ca2292cdc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045956 Sep 18 06:34:36.930041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1062/2066] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:37.411277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:37.411893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:37.417894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:37.418378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:37.418844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:37.419534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:37.419989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:37.420415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:37.549536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:37.557461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f9d22f89-298d-49ef-82ec-597a3a8c8fe8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150028 Sep 18 06:34:37.558022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1005/2067] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:37 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:37.878527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:37.880694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:37.886592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:37.887208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:37.887643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:37.888319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:37.888871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:37.889200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:37.919267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-08eff5ee-016f-4610-9d67-2986e90f9fb1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044679 Sep 18 06:34:37.919848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1063/2068] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:37 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:37.941738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:37.944038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:37.950825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:37.951313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:37.951744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:37.952430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:37.952878 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:37.953358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:37.983203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4cf0ce2c-dc1d-4e2f-b4c8-359d5cdc5854 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045654 Sep 18 06:34:37.983747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1006/2069] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:37 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:38.581942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:38.583004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:38.590268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:38.590789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:38.591381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:38.592592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:38.593360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:38.593999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:38.728080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:38.735821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9c3156a8-d99f-4469-af1d-9aafc31b6bc1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.158623 Sep 18 06:34:38.736381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1064/2070] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:38 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:38.932338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:38.935267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:38.943184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:38.943738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:38.944122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:38.944809 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:38.945262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:38.945687 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:38.973745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-006c9907-195a-468d-b7c9-39ddbe35478a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045492 Sep 18 06:34:38.974298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1007/2071] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:38 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:38.996477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:38.998646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:39.005030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:39.005517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:39.005960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:39.006675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:39.007134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:39.007567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:39.036375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-24687a4e-fc20-4b3e-aec9-1bd8f08f5c16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044891 Sep 18 06:34:39.036924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1065/2072] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:38 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:39.759047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:39.759559 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:39.765521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:39.766004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:39.766441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:39.767164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:39.767610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:39.768035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:39.892875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:39.900556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-865283af-6bc2-4f67-b1d2-5f1cb7c525f6 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.145591 Sep 18 06:34:39.901120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1008/2073] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:39 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:39.986806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:39.988954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:39.996147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:39.996628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:39.997198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:39.997725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:39.998171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:39.998614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:40.027953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-50ac0d5e-5761-41cf-80ce-ecd328011851 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045180 Sep 18 06:34:40.028435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1066/2074] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:39 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:40.049339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:40.051544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:40.057738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:40.058212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:40.058678 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:40.059359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:40.059813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:40.060230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:40.091041 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b09fc82e-4440-4dff-b76f-ec5a8f9f23e7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045606 Sep 18 06:34:40.091592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1009/2075] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:40.924062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:40.924713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:40.930760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:40.931244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:40.931679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:40.932365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:40.932819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:40.933246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:41.040424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:41.042561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:41.049193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:41.049669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:41.050226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:41.050825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:41.051267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:41.051670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:41.065266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:41.072988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-054d7de3-5057-4761-a8c6-b1e85fa0a7f5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152819 Sep 18 06:34:41.073548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1067/2076] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:40 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:41.081358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-de3ac2ba-8d8a-4ca0-b042-ae808a09c462 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044573 Sep 18 06:34:41.082052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1010/2077] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:41 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:41.103084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:41.105230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:41.111648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:41.112127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:41.112561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:41.113236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:41.113684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:41.114107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:41.143127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c5d1737e-dc82-4cb7-accf-d9e57b928f08 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043867 Sep 18 06:34:41.143662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1068/2078] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:42.094421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:42.096228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:42.096702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:42.096821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:42.103230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:42.103372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:42.103710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:42.103830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:42.104142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:42.104257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:42.104826 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:42.104938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:42.105275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:42.105375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:42.105726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:42.105827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:42.122195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:42.136310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0ea29ccc-c59e-46ed-84be-a7c8e8360f26 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046446 Sep 18 06:34:42.137401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1011/2079] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:42 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:42.155946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:42.158090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:42.163973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:42.164454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:42.164888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:42.165578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:42.166048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:42.166472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:42.195647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6c155d56-a167-4856-88fa-3c8536db1186 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044171 Sep 18 06:34:42.196429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1012/2080] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:42.235206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:42.242855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-21edb932-c79b-42ed-8f12-c88df3f8f8a8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150414 Sep 18 06:34:42.243426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1069/2081] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:42 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:34:43.149409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:43.151568 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:43.158444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:43.158953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:43.159383 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:43.160060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:43.160508 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:43.160929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:43.189494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-47f50d58-2cb2-4d3e-aa84-d3d8d3c3d061 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044096 Sep 18 06:34:43.190038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1013/2082] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:43 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:43.208765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:43.210967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:43.217866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:43.218344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:43.218816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:43.219497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:43.219951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:43.220372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:43.249695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-24b104a9-7db3-4184-84e5-8e23f7b5fd6b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044983 Sep 18 06:34:43.250239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1070/2083] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:43.267057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:43.267666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:43.273462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:43.273934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:43.274362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:43.275060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:43.275506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:43.275955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:43.404723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:43.412317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c1ad6832-787f-44eb-b3b4-53300358e9a2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149123 Sep 18 06:34:43.412875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1014/2084] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:43 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:44.201893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:44.204064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:44.210032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:44.210617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:44.210992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:44.211681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:44.212145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:44.212576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:44.241434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-160a711a-703e-417c-8df8-12c0de2370e6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043539 Sep 18 06:34:44.241982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1071/2085] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:44 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:44.261346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:44.263578 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:44.269894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:44.270374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:44.270845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:44.271527 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:44.271974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:44.272393 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:44.301480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9d6e5281-74ff-4a12-b134-13e481251838 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044169 Sep 18 06:34:44.302035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1015/2086] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:44.435809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:44.436413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:44.442325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:44.442902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:44.443336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:44.444009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:44.444454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:44.444873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:44.572673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:44.574867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:44.581550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b762a469-3bea-411e-82f8-a7965458431b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150171 Sep 18 06:34:44.582112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1072/2087] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:44 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:45.254246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:45.256431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:45.262599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:45.263164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:45.263503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:45.264177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:45.264626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:45.265047 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:45.293847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d7744366-6177-4062-8452-ea357ee54ab9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042982 Sep 18 06:34:45.294389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1016/2088] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:45 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:45.313241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:45.315434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:45.322466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:45.322974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:45.323407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:45.324082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:45.324546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:45.324951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:45.353891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3b3a88d7-9850-43f3-9ee1-144537bac4ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044588 Sep 18 06:34:45.354428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1073/2089] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:45 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:45.605335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:45.606541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:45.612585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:45.613074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:45.613517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:45.614207 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:45.614698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:45.615128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:45.745223 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:45.753023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3ae9a36e-0a1d-4957-a20c-a5d6d1c7ecdb tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151690 Sep 18 06:34:45.753581 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1017/2090] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:45 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:46.306190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:46.308352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:46.314389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:46.314993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:46.315319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:46.315995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:46.316442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:46.316867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:46.346797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-123fb6d7-f885-466c-8f2b-b844c95401c3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044629 Sep 18 06:34:46.347343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1074/2091] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:46 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:46.366718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:46.368858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:46.375970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:46.376450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:46.376883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:46.377565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:46.378013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:46.378430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:46.408100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0486a18e-116c-493b-9ec8-b8e519e04a20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045312 Sep 18 06:34:46.408641 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1018/2092] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:46.776592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:46.777226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:46.783185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:46.783658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:46.784089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:46.784761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:46.785212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:46.785628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:46.917965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:46.925701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a5907bf9-3900-44c6-b418-e9509b640240 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152925 Sep 18 06:34:46.926484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1075/2093] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:46 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:47.359899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:47.362069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:47.368127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:47.368700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:47.369021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:47.369953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:47.370405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:47.370856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:47.399681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-cad43f73-bc2b-4f05-bcac-9c74ef792827 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044297 Sep 18 06:34:47.400228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1019/2094] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:47 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:47.421185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:47.423348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:47.429753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:47.430225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:47.430683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:47.431360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:47.431805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:47.432222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:47.462495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bb025f84-8e18-4722-be45-d431167ef4b9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045607 Sep 18 06:34:47.463090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1076/2095] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:47.949340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:47.949954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:47.956631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:47.957115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:47.957561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:47.958240 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:47.958723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:47.959148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:48.084649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:48.092927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a3fd9f00-6c61-499a-9404-d7fc708704b7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147471 Sep 18 06:34:48.093481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1020/2096] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:47 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:48.412127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:48.414246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:48.420226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:48.420693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:48.421123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:48.421794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:48.422364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:48.422675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:48.451710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-43c7e00a-2ff3-4585-88c8-46c1f33edbf6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043510 Sep 18 06:34:48.452265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1077/2097] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:48 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:48.474769 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:48.476908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:48.483330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:48.483847 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:48.484285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:48.484974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:48.485428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:48.485853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:48.515597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1e3c1a9c-caee-45d7-a19f-39a0764fba23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044788 Sep 18 06:34:48.516148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1021/2098] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:49.115719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:49.116331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:49.122650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:49.123144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:49.123585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:49.124270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:49.124728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:49.125156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:49.253565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:49.261564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f3bf4a8f-440e-4559-a4ed-ecdbb250d4fa tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149683 Sep 18 06:34:49.262134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1078/2099] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:49 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:49.464538 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:49.467435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:49.474478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:49.474998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:49.475535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:49.476108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:49.476553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:49.477083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:49.508323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8f71d479-e7af-485a-bad8-f9f3a8e9f377 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047819 Sep 18 06:34:49.508880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1022/2100] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:49 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:49.527805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:49.530726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:49.537140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:49.537650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:49.538059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:49.538776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:49.539233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:49.539659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:49.568450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d1396037-35f0-42f5-94ce-68250bd00aca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044591 Sep 18 06:34:49.569014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1079/2101] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:50.284466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:50.285080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:50.291106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:50.291597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:50.292086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:50.292731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:50.293187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:50.293626 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:50.421445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:50.429191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2b7af073-4cbb-41b8-8b1e-d39737aa1c6f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148581 Sep 18 06:34:50.429760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1023/2102] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:50 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:50.521310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:50.523507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:50.530018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:50.530618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:50.530953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:50.531632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:50.532085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:50.532510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:50.561785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ea4a2ada-e599-4ba4-a510-da8a91e61354 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044396 Sep 18 06:34:50.562508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1080/2103] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:50 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:50.580107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:50.582278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:50.588627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:50.589106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:50.589536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:50.590215 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:50.590691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:50.591606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:50.622540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5b0a2c19-c75f-4ebc-9019-d82cb9c600eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046258 Sep 18 06:34:50.623128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1024/2104] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:51.453411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:51.454022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:51.459973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:51.460456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:51.460889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:51.461582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:51.462164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:51.462698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:51.573259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:51.575452 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:51.581853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:51.582438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:51.582793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:51.583514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:51.583967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:51.584389 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:51.590467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:51.598336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6289a89d-c010-4e5c-8335-930ed054da07 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149485 Sep 18 06:34:51.598938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1081/2105] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:51 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:51.613838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-67c9b149-52c0-4eb2-8f3b-b50ad6de2287 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044456 Sep 18 06:34:51.614380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1025/2106] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:51 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:51.634982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:51.637132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:51.643038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:51.643522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:51.643966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:51.644659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:51.645148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:51.645575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:51.676593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1b60a926-aa3c-4a6d-823f-8a8d6cf85381 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045529 Sep 18 06:34:51.677151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1082/2107] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:52.622241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:52.622917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:52.626080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:52.628248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:52.629364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:52.629857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:52.630297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:52.631020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:52.631484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:52.631912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:52.634539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:52.635045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:52.635476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:52.636157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:52.636609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:52.637031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:52.667804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1e265788-b786-4751-b868-6bba1c104b9d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045547 Sep 18 06:34:52.668663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1083/2108] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:52 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:52.688990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:52.691306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:52.697421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:52.697903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:52.698336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:52.699068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:52.699548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:52.699969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:52.732342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e1462a0a-6214-4187-90ae-b62883baa790 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047032 Sep 18 06:34:52.732838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1084/2109] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:52 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:53.019434 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:53.027032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-402a1e6b-0181-4f9c-b8a5-b41d454f3640 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.408953 Sep 18 06:34:53.027595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1026/2110] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:52 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:53.681202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:53.683407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:53.689834 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:53.690311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:53.690770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:53.691451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:53.691932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:53.692360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:53.720817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a9c9a40b-7d69-4ad3-a955-30be59fdfb95 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043590 Sep 18 06:34:53.721296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1085/2111] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:53 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:53.744015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:53.746432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:53.748759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=88 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:53.754018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:53.754510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:53.754975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:53.755669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:53.756120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:53.756542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:53.786446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-eed60351-e400-4ec5-ad72-835d3c98af02 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046279 Sep 18 06:34:53.787030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1027/2112] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:54.049790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:54.050312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:54.056206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:54.056681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:54.057114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:54.057799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:54.058246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:54.058695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:54.185356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:54.192887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-be49d33d-3c1c-45b5-bde1-5fe74813d7f2 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.147053 Sep 18 06:34:54.193447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1086/2113] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:54 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:54.733856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:54.735955 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:54.741867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:54.742346 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:54.742830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:54.743512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:54.744023 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:54.744698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:54.748930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:54.773474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-baa68dbf-0f8c-482e-b754-bbee6b3bd07d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044228 Sep 18 06:34:54.774018 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1028/2114] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:54 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:54.799424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:54.801554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:54.807941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:54.808416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:54.808847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:54.809521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:54.810627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:54.811051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:54.839364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-84615782-aa3b-4d11-a9cf-8906ea949609 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043841 Sep 18 06:34:54.839909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1087/2115] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:55.216825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:55.217439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:55.223419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:55.223902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:55.224339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:55.225029 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:55.225481 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:55.225901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:55.353432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:55.361831 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c1d40e85-4da8-4b99-972c-d21bdc609ab8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149339 Sep 18 06:34:55.362400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1029/2116] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:55 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:55.785953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:55.788166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:55.794544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:55.795061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:55.795494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:55.796181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:55.796634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:55.797059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:55.827112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ef120544-390e-4e78-be7c-0f12e949639f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045146 Sep 18 06:34:55.827741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1088/2117] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:55 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:55.851490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:55.853622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:55.859942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:55.860447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:55.860883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:55.861560 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:55.862009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:55.862428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:55.892480 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3de5cdac-1d08-4639-98b6-d244274c0f5a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044841 Sep 18 06:34:55.893015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1030/2118] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:56.385255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:56.385889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:56.391921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:56.392394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:56.392828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:56.393503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:56.393951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:56.394370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:56.408821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:56.522665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:56.530927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-80f5a235-1e38-4b0c-87ff-aeba83750b4c tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149498 Sep 18 06:34:56.531494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1089/2119] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:56 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:56.839086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:56.841730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:56.847695 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:56.848203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:56.848636 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:56.849314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:56.849766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:56.850191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:56.880376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-13535d2d-10df-471b-b50f-16513d8349a3 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045203 Sep 18 06:34:56.880987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1031/2120] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:56 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:56.905307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:56.907499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:56.913817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:56.914293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:56.914767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:56.915444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:56.915892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:56.916345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:56.951189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-490674d1-7c28-4109-9f1b-a96c0307815d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049841 Sep 18 06:34:56.952154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1090/2121] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:56 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:57.554859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:57.555503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:57.561894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:57.562377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:57.562836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:57.563525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:57.563974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:57.564401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:57.696245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:57.703819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e99c208b-2c82-4282-a903-3edf275cf1ce tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152983 Sep 18 06:34:57.704380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1032/2122] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:57 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:57.893061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:57.895219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:57.901167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:57.901649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:57.902088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:57.902809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:57.903269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:57.903694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:57.932498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-28fe084b-6560-4adf-a0e9-0e1386a9a89e tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043445 Sep 18 06:34:57.933045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1091/2123] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:57 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:57.963417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:57.965547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:57.972589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:57.973066 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:57.973498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:57.974206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:57.974679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:57.975099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:58.004410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ef350016-4bf4-4c3a-92e9-25aa5160b462 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044877 Sep 18 06:34:58.004944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1033/2124] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:58.726759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:58.727289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:58.733226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:58.733705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:58.734143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:58.734858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:58.735312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:58.735736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:58.861668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:34:58.869433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6b86d175-e234-492f-971f-ec4504dd9522 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.146678 Sep 18 06:34:58.870257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1092/2125] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:58 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:34:58.945516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:58.947709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:58.954177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:58.954688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:58.955127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:58.955813 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:58.956263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:58.956686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:58.985191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-443a4f34-98b8-425e-b958-e46bd0d3ca3d tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043539 Sep 18 06:34:58.985804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1034/2126] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:58 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:34:59.017422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:59.019591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:59.025955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:59.026433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:59.026911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:59.027597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:59.028051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:59.028473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:59.056915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-44d19026-2bea-478c-902d-d232fd84963b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044185 Sep 18 06:34:59.057461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1093/2127] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:59 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:34:59.892645 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:34:59.893165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:34:59.899000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:59.899483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:59.899914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:59.900590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:34:59.901039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:34:59.901457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:34:59.997556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:00.000651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:00.002784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=84 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:00.008392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:00.009017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:00.009483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:00.010188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:00.010658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:00.011110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:00.033582 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:00.040430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2d22968e-3ce5-4a46-aed0-44b4bc311422 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047022 Sep 18 06:35:00.040992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1094/2128] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:34:59 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:00.041242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-55a9774e-918c-4245-8d39-7b98ac48ab37 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152623 Sep 18 06:35:00.041799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1035/2129] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:34:59 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:35:00.069178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:00.071348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:00.077202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:00.077679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:00.078109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:00.078816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:00.079264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:00.079687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:00.108940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9744eddd-0be5-403e-8223-7c7fd1400439 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043381 Sep 18 06:35:00.109482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1095/2130] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:01.053449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:35:01.055646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:01.062545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:01.063059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:01.063499 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:01.064187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:01.064638 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:01.064955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:01.065376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:01.065560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:01.071825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:01.072305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:01.072733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:01.073411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:01.073866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:01.074283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:01.142150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-14fb763f-d3e9-41a3-986f-9fade5ce0bd6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/action" status: 200 len: 82238 microversion: 2.1 time: 0.092402 Sep 18 06:35:01.143276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1036/2131] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:35:01 2022] POST /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/action => generated 82238 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:35:01.148198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:01.150363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:01.157151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:01.157629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:01.158061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:01.158779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:01.159229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:01.159649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:01.190313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-352fb5fc-78aa-4d99-8fa8-9ad973f730f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046152 Sep 18 06:35:01.190899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1037/2132] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:01.195365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:01.195844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:01.202393 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:01.202902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:01.203333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:01.204035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:01.204483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:01.204902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:01.212877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:01.220691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-19133f2d-4f66-4384-a228-da3e76810d9d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.160010 Sep 18 06:35:01.221247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1096/2133] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:01 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:02.203354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:02.205528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:02.211969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:02.212448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:02.212907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:02.213582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:02.214030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:02.214497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:02.244168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-23901231-4aa9-4d59-8cfa-60ad793da30f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045326 Sep 18 06:35:02.244744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1097/2134] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:02.249404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:02.249991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:02.255313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:02.255787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:02.256214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:02.256876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:02.257322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:02.257742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:02.383392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:02.391599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3a79f59e-4a8a-45cb-b5bf-97121b03aaf9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.145655 Sep 18 06:35:02.392187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1098/2135] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:02 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:35:03.257348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:03.259627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:03.266049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:03.266540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:03.267033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:03.267708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:03.268154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:03.268577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:03.299828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-375bd900-74bd-4371-8a48-ea6ac8b4a148 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046386 Sep 18 06:35:03.300741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1099/2136] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:03.415027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:03.415628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:03.421422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:03.422439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:03.422986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:03.423691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:03.424266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:03.424688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:03.555425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:03.563668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1b175003-70a0-4406-b179-b29d8a954fd7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152443 Sep 18 06:35:03.564239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1100/2137] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:03 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:04.313662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:04.315898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:04.322506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:04.323113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:04.323551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:04.324227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:04.324677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:04.325097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:04.354191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bb06846a-1c95-4913-af86-a0c8a0e8c8ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045198 Sep 18 06:35:04.354760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1101/2138] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:04 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:04.587192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:04.587800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:04.593658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:04.594681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:04.595111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:04.595785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:04.596233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:04.596651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:04.724191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:04.731886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2107b005-479d-4d0a-83c1-7e186b49fdef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.148557 Sep 18 06:35:04.732450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1102/2139] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:04 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:05.367159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:05.369352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:05.376096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:05.376581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:05.377018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:05.377704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:05.378162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:05.378596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:05.407180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9f40d17b-7cae-4781-872b-d343b98fc4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043952 Sep 18 06:35:05.408221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1103/2140] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:05.757026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:05.757637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:05.763597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:05.764076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:05.764509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:05.765193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:05.765646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:05.766071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:05.900241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:05.907906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6817792b-4aa0-4fcf-bcf0-9e0e3eb860a1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155258 Sep 18 06:35:05.908470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1104/2141] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:05 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:06.419970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:06.422115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:06.428593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:06.429067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:06.429503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:06.430176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:06.430651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:06.431075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:06.460917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f82b66c5-ef88-40f7-baf9-51d00ee3f9c1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044820 Sep 18 06:35:06.461480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1105/2142] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:06.930945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:06.931552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:06.937425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:06.937902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:06.938331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:06.939035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:06.939482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:06.939897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:06.942367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:07.069415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:07.076978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-690b8946-986e-4515-8ac3-f65fd5c90540 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149878 Sep 18 06:35:07.077548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1106/2143] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:06 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:07.473471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:07.475661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:07.482020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:07.482509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:07.482978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:07.483661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:07.484117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:07.484542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:07.514034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-56d78088-869b-4ad8-ab05-a815cf9b0b4e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044568 Sep 18 06:35:07.514597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1107/2144] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:08.100964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:08.101576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:08.107578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:08.108059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:08.108496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:08.109173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:08.109630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:08.110053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:08.237587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:08.246077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1704baae-040f-464f-a68a-1c27d33a1f3b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149547 Sep 18 06:35:08.246661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1108/2145] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:08 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:08.526337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:08.528601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:08.535070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:08.536234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:08.536676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:08.537371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:08.537829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:08.538260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:08.567462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b3954773-9ce2-40fe-ba73-109738220e20 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045064 Sep 18 06:35:08.568021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1109/2146] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:09.269310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:09.269914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:09.276575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:09.277061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:09.277499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:09.278184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:09.278662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:09.279087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:09.412792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:09.420505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0288376a-b09b-48f3-a006-2e4aedcc8ea1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155107 Sep 18 06:35:09.421069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1110/2147] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:09 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:09.579949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:09.582025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:09.589688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:09.590175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:09.590629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:09.591325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:09.591778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:09.592200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:09.626361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b8ba9326-ad7e-4135-acff-b4484d336c64 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050363 Sep 18 06:35:09.626954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1111/2148] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:10.444534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:10.445131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:10.451149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:10.451630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:10.452065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:10.452752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:10.453201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:10.453617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:10.583115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:10.592020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ec7be000-f85f-4371-a676-780f6c263b31 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151839 Sep 18 06:35:10.592595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1112/2149] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:10 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:10.638155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:10.640332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:10.645941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:10.646451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:10.646923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:10.647635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:10.648087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:10.648515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:10.677039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-054c0bda-a0ee-40f4-bf24-03896e6bdc35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042515 Sep 18 06:35:10.678119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1113/2150] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:35:11.200317 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=23 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:11.222177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=60 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:11.615042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:11.615658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:11.622386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:11.622918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:11.623338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:11.624023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:11.624477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:11.624903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:11.758589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:11.766293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-04a744a6-2dfe-4834-9f7b-f3e43c10b82b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155118 Sep 18 06:35:11.766905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1114/2151] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:11 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:11.772020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:11.774434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:11.780141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:11.780615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:11.781072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:11.781749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:11.782195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:11.782649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:11.810696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d83a1d60-add0-49d9-b1b1-5baddb2ba2f7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042780 Sep 18 06:35:11.811235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1115/2152] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:12.790831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:12.791419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:12.798031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:12.798516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:12.799003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:12.799726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:12.800228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:12.800657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:12.928982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:12.936610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d84c5694-3f3a-4f23-9750-b62080ff6972 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.150257 Sep 18 06:35:12.937173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1116/2153] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:12 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:12.941630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:12.943770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:12.949414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:12.949895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:12.950330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:12.951055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:12.951659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:12.952084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:12.981679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-50915007-dd76-4edf-9c4e-e079167684fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043524 Sep 18 06:35:12.982219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1117/2154] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:13.959629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:13.960276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:13.966673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:13.967154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:13.967586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:13.968277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:13.968715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:13.969142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:14.100404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:14.108062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e2ccd4ed-71f4-4177-99c1-d18a9591dc5a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.152311 Sep 18 06:35:14.108632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1118/2155] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:13 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:14.113064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:14.115798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:14.121535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:14.122016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:14.122453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:14.123159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:14.123615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:14.124043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:14.152132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc016622-f6fa-4493-a644-1bc226745083 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042541 Sep 18 06:35:14.152675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1119/2156] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:15.132662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:15.133308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:15.139837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:15.140319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:15.140762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:15.141447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:15.141901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:15.142329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:15.269740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:15.277961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-55b27da1-9c9f-4655-be6d-c880502f6843 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149382 Sep 18 06:35:15.278535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1120/2157] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:15 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:15.282984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:15.285083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:15.291401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:15.291880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:15.292311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:15.293017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:15.293462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:15.293882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:15.322838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-90bcad50-df1b-4e81-8b9c-7cabee033166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043259 Sep 18 06:35:15.323389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1121/2158] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:16.301842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:16.302447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:16.308923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:16.309393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:16.309815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:16.310490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:16.310956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:16.311372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:16.438312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:16.440474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:16.447758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9c478277-ae85-4e25-b5ee-c202913b0f69 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.149882 Sep 18 06:35:16.448326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1122/2159] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:16 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:16.452902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:16.455034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:16.461500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:16.461977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:16.462415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:16.463136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:16.463583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:16.464005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:16.492799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ec97a87a-799a-49bb-911d-7a699c8bdeab tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043444 Sep 18 06:35:16.493354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1123/2160] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:17.471608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:17.472217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:17.479471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:17.479948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:17.480383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:17.481067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:17.481507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:17.481923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:17.614687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:17.622640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1cbae8d4-5e37-4925-ae9a-a23c4a98ec49 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154896 Sep 18 06:35:17.623206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1124/2161] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:17 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:17.627631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:17.629693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:17.635120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:17.635593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:17.636035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:17.636704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:17.637153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:17.637571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:17.666669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-506ef7f0-f2b9-418c-b4d7-7254ab2dc254 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042482 Sep 18 06:35:17.667211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1125/2162] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:18.646904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:18.649116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:18.657444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:18.657918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:18.658354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:18.659068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:18.659515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:18.659940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:18.789612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:18.797289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9b68f22a-dabc-4c10-be58-7dd2c15695ef tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.155907 Sep 18 06:35:18.797873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1126/2163] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:18 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:18.802344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:18.805089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:18.810557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:18.811064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:18.811495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:18.812171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:18.812620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:18.813042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:18.841720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-97f7a85f-c3c8-4cdc-bb23-6ad9745dc823 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042875 Sep 18 06:35:18.842267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1127/2164] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:19.821281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:19.821953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:19.828407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:19.828891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:19.829323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:19.830006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:19.830453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:19.830909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:19.967735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:19.975914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2806ca73-6002-41b8-b8ce-3a5b07091ca3 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.159180 Sep 18 06:35:19.976473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1128/2165] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:19 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:19.981143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:19.983250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:19.988745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:19.989223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:19.989657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:19.990387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:19.991376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:19.991832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:20.020910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ffdf67ed-d9b6-4c41-87de-59ca60bc9d8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043426 Sep 18 06:35:20.021463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1129/2166] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:20.999955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:21.000633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:21.007715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:21.008191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:21.008648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:21.009330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:21.009776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:21.010190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:21.139552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:21.147434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0157c710-a0ae-4bba-b0c2-a70c53c8b9b8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.151717 Sep 18 06:35:21.148006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1130/2167] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:20 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:21.152567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:21.154705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:21.160414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:21.160899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:21.161334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:21.162011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:21.162462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:21.162917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:21.192399 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-662c94ca-b462-495d-a0fb-ecb36e904630 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043400 Sep 18 06:35:21.192953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1131/2168] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:22.170473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:22.171205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:22.177684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:22.178193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:22.178760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:22.179456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:22.179913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:22.180342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:22.312900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:35:22.325223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b889287a-a251-4a55-a52f-4073b3a05785 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.158627 Sep 18 06:35:22.325886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1132/2169] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:35:22 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:35:22.330510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:22.332725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:22.339994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:22.340477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:22.341967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:22.342703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:22.343607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:22.344036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:22.364738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:35:22.394545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-39c47f93-a020-49dd-a3db-c218b164d127 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067558 Sep 18 06:35:22.399150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1133/2170] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:35:22 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:35:22.399936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:35:22.400519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:35:22.406137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:22.406686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:22.407124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:22.407817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:35:22.408271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:35:22.408722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:35:32.379214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:32.403447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=81 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:32.429772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=71 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:36:05.241189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:36:05.241781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:36:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:36:07.748836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b4759db2-5419-4562-a352-c04b01426c0e) Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:36:07.751107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:36:07.752784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b4759db2-5419-4562-a352-c04b01426c0e) Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:36:07.753599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b4759db2-5419-4562-a352-c04b01426c0e) Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:36:07.755188 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b4759db2-5419-4562-a352-c04b01426c0e) Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:36:07.757078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b4759db2-5419-4562-a352-c04b01426c0e) Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:36:07.758404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b4759db2-5419-4562-a352-c04b01426c0e) Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b4759db2-5419-4562-a352-c04b01426c0e) Sep 18 06:36:07.760436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.761703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b4759db2-5419-4562-a352-c04b01426c0e) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:36:07.761703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2b33cb03-c583-4803-9a55-2bc1e4a86777 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 66.560710 Sep 18 06:36:07.761703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1038/2171] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:35:01 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 269 bytes in 66562 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:36:07.761703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:36:07.762087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:36:07.765766 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:36:07.766253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:36:07.766715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:36:07.767404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:36:07.767965 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:36:07.768262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:36:07.856582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] HTTP exception thrown: Cannot 'detach_volume' instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 while it is in task_state spawning Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Request method failure captured: Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments/15273f08-c5c3-4dc4-9f2f-b809495a2590 HTTP/1.1 Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrn1lB3_UfTsYaV5VDBK-ypfzlj8urkIAwTPw5Yd2WBNRG2q1UHomYg5YU0Q_Gl1eVXe7O3KzhOZQBsmLKDSwprGT0ibDlrMshfvA4R7vtAqET5IlKrB0SXuHk2TN8wUSFWEaynYf04KL8MaFpR88nkIfYDjj1PbzrH7W__S3Z_5VEc Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:36:07.857508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 00399347c76d451d94d45f4edc3cf416 Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:36:07.859223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Cannot 'detach_volume' instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 while it is in task_state spawning Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': '5ed3cfa3-6093-4c26-9877-01470acd2e16', 'id': '15273f08-c5c3-4dc4-9f2f-b809495a2590'} Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 155, in inner Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 in task_state spawning. Cannot detach_volume while the instance is in this state. Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:36:07.864803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 while it is in task_state spawning Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 while it is in task_state spawning Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:36:07.866352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Returning 409 to user: Cannot 'detach_volume' instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 while it is in task_state spawning {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:36:07.867635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-744e872e-726b-4213-a5bf-c8a410afba4f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "DELETE /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments/15273f08-c5c3-4dc4-9f2f-b809495a2590" status: 409 len: 155 microversion: 2.20 time: 45.462803 Sep 18 06:36:07.867635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1134/2172] 10.222.0.20 () {60 vars in 1457 bytes} [Sun Sep 18 06:35:22 2022] DELETE /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments/15273f08-c5c3-4dc4-9f2f-b809495a2590 => generated 155 bytes in 45464 msecs (HTTP/1.1 409) 9 headers in 380 bytes (1 switches on core 0) Sep 18 06:36:07.867635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:36:07.867635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:36:07.870639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:36:07.871112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:36:07.871544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:36:07.872545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:36:07.872669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:36:07.873072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:36:08.007424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:36:08.015069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9aff12fa-6283-4038-a889-5ca9a80f7ec1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1570 microversion: 2.1 time: 0.154726 Sep 18 06:36:08.015637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1135/2173] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:36:07 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1570 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:36:08.020059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:36:08.022239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:36:08.027926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:36:08.028408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:36:08.028844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:36:08.029535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:36:08.029992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:36:08.030445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:36:08.062742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-deb3537c-2b28-4063-8191-bada858b5819 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046039 Sep 18 06:36:08.063301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1136/2174] 10.222.0.20 () {58 vars in 1302 bytes} [Sun Sep 18 06:36:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:36:08.067877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:36:08.068362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:36:08.074023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:36:08.074561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:36:08.075028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:36:08.075839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:36:08.076344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:36:08.076766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:36:17.765468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:36:17.786394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:36:18.071425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=6 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:36:18.094462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=18 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:36:25.321120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:36:25.321911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:36:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:36:26.374940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:36:26.375517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:36:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:37:22.597789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:37:22.638777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:37:32.602286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:37:32.650471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:37:48.462449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:37:48.463103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:37:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:38:06.380078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:38:06.380661 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:38:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:38:08.022169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9489918f-5bf7-4835-a184-9d95942e21ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 202 len: 0 microversion: 2.1 time: 120.267720 Sep 18 06:38:08.022869 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1039/2175] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:36:07 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 0 bytes in 120269 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:38:08.028509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:38:08.031510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:38:08.037834 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.038374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.038892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.039587 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.040031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.040454 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.047190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b62936b4-0eae-497e-aed3-048a35398aee) Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:38:08.050400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:38:08.051863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b62936b4-0eae-497e-aed3-048a35398aee) Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:38:08.052508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b62936b4-0eae-497e-aed3-048a35398aee) Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:38:08.053596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b62936b4-0eae-497e-aed3-048a35398aee) Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:38:08.056146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b62936b4-0eae-497e-aed3-048a35398aee) Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:38:08.060060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b62936b4-0eae-497e-aed3-048a35398aee) Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b62936b4-0eae-497e-aed3-048a35398aee) Sep 18 06:38:08.061434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:38:08.066568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b62936b4-0eae-497e-aed3-048a35398aee) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:38:08.066568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-079b33b6-5d89-47b8-b866-9ab066fdab06 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 119.987786 Sep 18 06:38:08.066568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1137/2176] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:36:08 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 269 bytes in 119989 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:38:08.072290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f9144c59-b6e9-4252-bd14-cddacdd3542c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048250 Sep 18 06:38:08.073155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1040/2177] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:38:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:38:08.224309 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-172b4547-dae9-4b00-b3c6-951c37db70c0 req-eff6e2e2-aae0-4a90-9354-c0e7c730e362 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ce28d02-87b5-461d-bac3-2bf15b26fea9", "tag": "726ec9db-ceda-4200-b057-55e60adeb8a9"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:38:08.234227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-172b4547-dae9-4b00-b3c6-951c37db70c0 req-eff6e2e2-aae0-4a90-9354-c0e7c730e362 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.234566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-172b4547-dae9-4b00-b3c6-951c37db70c0 req-eff6e2e2-aae0-4a90-9354-c0e7c730e362 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.235165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-172b4547-dae9-4b00-b3c6-951c37db70c0 req-eff6e2e2-aae0-4a90-9354-c0e7c730e362 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.246745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-172b4547-dae9-4b00-b3c6-951c37db70c0 req-eff6e2e2-aae0-4a90-9354-c0e7c730e362 service nova] Creating event network-changed:726ec9db-ceda-4200-b057-55e60adeb8a9 for instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 on hv-cinder-85535 Sep 18 06:38:08.253108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-172b4547-dae9-4b00-b3c6-951c37db70c0 req-eff6e2e2-aae0-4a90-9354-c0e7c730e362 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.177981 Sep 18 06:38:08.253896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1041/2178] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:38:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:38:08.259153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:38:08.261538 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:38:08.267391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:38:08.268037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:38:08.268466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.268969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.269415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.270310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.270811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.271247 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.277102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.277840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.278271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.279097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.279713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.280571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.320496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:38:08.367042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d884ab9d-a110-407f-b487-11a77a79f0b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "POST /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/action" status: 200 len: 76409 microversion: 2.1 time: 0.111578 Sep 18 06:38:08.367923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1042/2179] 10.222.0.20 () {60 vars in 1278 bytes} [Sun Sep 18 06:38:08 2022] POST /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e/action => generated 76409 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:38:08.373132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:38:08.373681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:38:08.381148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.381948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.382387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.383113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.383577 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.384101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.404133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-04f7ef3c-0f43-416d-a172-3e60f92d7769 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 204 len: 0 microversion: 2.1 time: 0.349751 Sep 18 06:38:08.404831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1138/2180] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:38:08 2022] DELETE /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:38:08.411880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:38:08.412706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:38:08.418933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.419668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.420271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:08.421134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:38:08.421784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:38:08.422383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:38:18.380982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=8 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:38:18.416137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=17,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:38:18.418565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:38:18.441643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:39:22.951915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:39:23.048713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:39:25.251349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:39:25.251898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:39:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:39:32.956982 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:39:33.053966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:39:45.331892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:39:45.332536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:39:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:39:46.389548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:39:46.390092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:39:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:40:08.351014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3de349e8-f04d-4737-a61f-5003b596db48) Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:40:08.353942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:40:08.355520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3de349e8-f04d-4737-a61f-5003b596db48) Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.357246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3de349e8-f04d-4737-a61f-5003b596db48) Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.358667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3de349e8-f04d-4737-a61f-5003b596db48) Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.360048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3de349e8-f04d-4737-a61f-5003b596db48) Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.361592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3de349e8-f04d-4737-a61f-5003b596db48) Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3de349e8-f04d-4737-a61f-5003b596db48) Sep 18 06:40:08.363145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.364442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3de349e8-f04d-4737-a61f-5003b596db48) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:08.364442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-77820fd5-c55b-4280-b233-b284a3383af4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 119.986464 Sep 18 06:40:08.364442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1043/2181] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:38:08 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 269 bytes in 119988 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:40:08.364442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.365074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.371899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.372382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.372861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.373586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.373981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.374492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.385651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-721c04ea-bcd0-47ba-b72b-5f737b1b2b68) Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:40:08.388043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:40:08.389434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-721c04ea-bcd0-47ba-b72b-5f737b1b2b68) Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.390771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-721c04ea-bcd0-47ba-b72b-5f737b1b2b68) Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.393483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-721c04ea-bcd0-47ba-b72b-5f737b1b2b68) Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.395212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-721c04ea-bcd0-47ba-b72b-5f737b1b2b68) Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.397225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-721c04ea-bcd0-47ba-b72b-5f737b1b2b68) Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-721c04ea-bcd0-47ba-b72b-5f737b1b2b68) Sep 18 06:40:08.400260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.402654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-721c04ea-bcd0-47ba-b72b-5f737b1b2b68) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:08.402654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7bc55eee-145b-4389-9e30-21ed3026866a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 119.982481 Sep 18 06:40:08.402654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1139/2182] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:38:08 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 269 bytes in 119985 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:40:08.402654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.403688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.405463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.405950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.406392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.407451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.407636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.407974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.413640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-37730692-9f30-4484-bfd4-b6e1513ac946 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055145 Sep 18 06:40:08.413640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1044/2183] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:40:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:40:08.418462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.419074 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.424772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.425349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.425691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.426374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.426864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.427290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.443896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:40:08.484237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Instance is already in deleting state, ignoring this request Sep 18 06:40:08.485133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1306bb2a-9e4e-44e1-a92a-8e8ac412747f tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 204 len: 0 microversion: 2.1 time: 0.092341 Sep 18 06:40:08.485898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1140/2184] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:40:08.490266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.490776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.496212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.496696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.497127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.497800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.498254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.498699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.512363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:40:08.513915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.515318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:08.516086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.517370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.520889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:08.520889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c66f01e8-5826-4b5a-9113-3617daf8d50e tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.028515 Sep 18 06:40:08.520889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1141/2185] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:40:08.520889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.520889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.527808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.528371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.528968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.529651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.530172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.530699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.546520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:40:08.623169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:40:08.651807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a91fb316-2e92-4c9e-9e28-b1dbb749f0e0) Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:40:08.654984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:40:08.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a91fb316-2e92-4c9e-9e28-b1dbb749f0e0) Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.657304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a91fb316-2e92-4c9e-9e28-b1dbb749f0e0) Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.658520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a91fb316-2e92-4c9e-9e28-b1dbb749f0e0) Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.659917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a91fb316-2e92-4c9e-9e28-b1dbb749f0e0) Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.661411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a91fb316-2e92-4c9e-9e28-b1dbb749f0e0) Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a91fb316-2e92-4c9e-9e28-b1dbb749f0e0) Sep 18 06:40:08.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.665002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a91fb316-2e92-4c9e-9e28-b1dbb749f0e0) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:08.666027 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-34ac4783-b5ea-41df-9a9e-b34a041658e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 0.252476 Sep 18 06:40:08.667286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1045/2186] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 269 bytes in 254 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:40:08.672381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.674729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.682659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.683329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.683767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.684459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.684906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.685325 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.704374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-23575a2c-820e-4a09-b677-f86e9423df02) Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:40:08.706627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:40:08.708231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-23575a2c-820e-4a09-b677-f86e9423df02) Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.709151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-23575a2c-820e-4a09-b677-f86e9423df02) Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.710469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-23575a2c-820e-4a09-b677-f86e9423df02) Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.712456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-23575a2c-820e-4a09-b677-f86e9423df02) Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.713783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-23575a2c-820e-4a09-b677-f86e9423df02) Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-23575a2c-820e-4a09-b677-f86e9423df02) Sep 18 06:40:08.715930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.717361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-23575a2c-820e-4a09-b677-f86e9423df02) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:08.717361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-51aa3507-4167-46da-8bc2-53d98f78eb65 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 0.191931 Sep 18 06:40:08.717361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1142/2187] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 269 bytes in 193 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:40:08.717361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.717726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.728977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.729475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.730035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.731045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.731497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.732133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.737087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c81c469f-7998-4ddd-9d87-6ece04d235c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068563 Sep 18 06:40:08.737756 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1046/2188] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:40:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:40:08.743550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.744152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.750284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.751067 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.751502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.752203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.753861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.753861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.773053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:40:08.851804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:40:08.867482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-46dd673c-871d-45d5-9b42-924a4d6fc905 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 204 len: 0 microversion: 2.1 time: 0.156546 Sep 18 06:40:08.868587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1143/2189] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:40:08.874845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.876093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.885551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.886492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.887144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.887873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.888299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.888734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.905844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:40:08.908219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.909861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:08.910791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:08.912271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.914782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:08.915988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d10af463-82c3-4d84-9294-5a54056d43e8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.045978 Sep 18 06:40:08.917059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1144/2190] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:40:08.922008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.922497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.928532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.928943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.929377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.930395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.930901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.931334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.931548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e0b18bc8-db48-46b6-a766-447598bb4cc9) Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:40:08.933750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:40:08.935105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e0b18bc8-db48-46b6-a766-447598bb4cc9) Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.935726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e0b18bc8-db48-46b6-a766-447598bb4cc9) Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:08.938934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e0b18bc8-db48-46b6-a766-447598bb4cc9) Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.940303 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e0b18bc8-db48-46b6-a766-447598bb4cc9) Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.941629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e0b18bc8-db48-46b6-a766-447598bb4cc9) Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e0b18bc8-db48-46b6-a766-447598bb4cc9) Sep 18 06:40:08.942914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:40:08.946460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e0b18bc8-db48-46b6-a766-447598bb4cc9) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:08.946460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1b930eef-dd15-457a-8117-c6823068580d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 0.196278 Sep 18 06:40:08.946460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1047/2191] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 269 bytes in 197 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:40:08.946460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.946893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.951211 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.951690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.952114 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.952815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.953289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.953691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.985301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-88b1b70e-71b3-4acb-81fa-392420c09626 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048233 Sep 18 06:40:08.986160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1048/2192] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:40:08 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:40:08.990687 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:08.991187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:08.997720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.998202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:08.998663 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:08.999359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:08.999812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:09.000237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:09.026729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Going to try to terminate instance {{(pid=120380) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:40:09.032694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:40:09.081376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.compute.api [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Instance is already in deleting state, ignoring this request Sep 18 06:40:09.082404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2a3c32a3-3d09-4cb0-944e-cf7520c625ec tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 204 len: 0 microversion: 2.1 time: 0.095106 Sep 18 06:40:09.082899 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1049/2193] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:40:09.091883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:09.092376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:09.098862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:09.099362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:09.099805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:09.100513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:09.100973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:09.101512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:09.103184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8b1e1354-4a7c-4316-a5ec-898c3f93a24b) Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:40:09.105351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:40:09.107054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8b1e1354-4a7c-4316-a5ec-898c3f93a24b) Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:09.107893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8b1e1354-4a7c-4316-a5ec-898c3f93a24b) Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:40:09.109001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8b1e1354-4a7c-4316-a5ec-898c3f93a24b) Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:09.110208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8b1e1354-4a7c-4316-a5ec-898c3f93a24b) Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:09.111422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8b1e1354-4a7c-4316-a5ec-898c3f93a24b) Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8b1e1354-4a7c-4316-a5ec-898c3f93a24b) Sep 18 06:40:09.112960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.115249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8b1e1354-4a7c-4316-a5ec-898c3f93a24b) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:09.116602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ad934c24-b022-4cc1-98dc-4e6c03d53bdf tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 0.198446 Sep 18 06:40:09.118154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1145/2194] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:40:08 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 269 bytes in 200 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:40:09.123387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:09.123892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:09.130065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:09.130558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:09.131343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:09.132046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:09.132503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:09.133537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:09.154015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:40:09.155787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:09.157421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:40:09.158394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:40:09.159880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:40:09.161259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:40:09.161259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cff941c3-ef2a-47ab-bbca-71defcbb8889 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.038062 Sep 18 06:40:09.161259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1146/2195] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:40:09 2022] DELETE /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:40:09.164791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:40:09.165316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:40:09.171392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:09.171876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:09.172506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:09.173375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:40:09.174423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:40:09.175036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:40:09.209307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:40:19.096409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=14 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:40:19.169034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=21 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:40:19.199373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=53,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:40:19.214961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=55,nova_cell1:UPDATE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:41:08.471849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:41:08.472406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:41:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:41:23.558953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:41:23.587621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d2227a51-78f7-47bb-8bac-efd9f95b13fc) Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:41:23.589781 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:41:23.591969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d2227a51-78f7-47bb-8bac-efd9f95b13fc) Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:41:23.592692 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d2227a51-78f7-47bb-8bac-efd9f95b13fc) Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:41:23.593877 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d2227a51-78f7-47bb-8bac-efd9f95b13fc) Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.596334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d2227a51-78f7-47bb-8bac-efd9f95b13fc) Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:41:23.597619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d2227a51-78f7-47bb-8bac-efd9f95b13fc) Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d2227a51-78f7-47bb-8bac-efd9f95b13fc) Sep 18 06:41:23.599165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.600613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d2227a51-78f7-47bb-8bac-efd9f95b13fc) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:41:23.600613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-990a17cd-49f9-4b21-8771-2c2298b9e075 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 74.507481 Sep 18 06:41:23.600613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1050/2196] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:40:09 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 269 bytes in 74509 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:41:23.710241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:41:23.713073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:41:23.719722 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:23.720199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:23.720632 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:23.721315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:23.721760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:23.722180 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:23.755077 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9f52a517-f48b-474a-83bc-f31a1dc25605 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.161058 Sep 18 06:41:23.755640 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1051/2197] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:41:23 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 197 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:41:23.761001 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:41:23.761936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:41:23.768015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:23.768721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:23.769152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:23.769841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:23.770293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:23.770744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:23.837096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:41:23.845584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-668c44e9-c920-4fbd-b16a-758ac592285b tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1220 microversion: 2.1 time: 0.088803 Sep 18 06:41:23.846128 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1052/2198] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:41:23 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:41:23.851805 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-ece2caf3-951d-462d-bf2e-197cf1215e3e req-432753cf-aab4-451c-a9ae-ce75142b763e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ce28d02-87b5-461d-bac3-2bf15b26fea9", "name": "network-vif-deleted", "tag": "726ec9db-ceda-4200-b057-55e60adeb8a9"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:41:23.861794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ece2caf3-951d-462d-bf2e-197cf1215e3e req-432753cf-aab4-451c-a9ae-ce75142b763e service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:23.862276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ece2caf3-951d-462d-bf2e-197cf1215e3e req-432753cf-aab4-451c-a9ae-ce75142b763e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:23.862736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ece2caf3-951d-462d-bf2e-197cf1215e3e req-432753cf-aab4-451c-a9ae-ce75142b763e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:23.875501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-ece2caf3-951d-462d-bf2e-197cf1215e3e req-432753cf-aab4-451c-a9ae-ce75142b763e service nova] Creating event network-vif-deleted:726ec9db-ceda-4200-b057-55e60adeb8a9 for instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 on hv-cinder-85535 Sep 18 06:41:23.879983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR oslo.messaging._drivers.impl_rabbit [req-ece2caf3-951d-462d-bf2e-197cf1215e3e req-432753cf-aab4-451c-a9ae-ce75142b763e service nova] [30a3a4b1-8749-43a6-8164-1025dc424aa6] AMQP server on 10.222.0.20:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 18 06:41:23.925410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3041a70f-d5dc-44a0-9969-c7dd6371ec06) Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:41:23.927739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:41:23.929307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3041a70f-d5dc-44a0-9969-c7dd6371ec06) Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:41:23.930369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3041a70f-d5dc-44a0-9969-c7dd6371ec06) Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:41:23.932008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3041a70f-d5dc-44a0-9969-c7dd6371ec06) Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.933428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3041a70f-d5dc-44a0-9969-c7dd6371ec06) Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:41:23.934871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3041a70f-d5dc-44a0-9969-c7dd6371ec06) Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3041a70f-d5dc-44a0-9969-c7dd6371ec06) Sep 18 06:41:23.936231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:23.937706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3041a70f-d5dc-44a0-9969-c7dd6371ec06) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:41:23.937706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e6ce8184-02d3-400f-ba96-dcea7341baeb tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 74.768838 Sep 18 06:41:23.937706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1147/2199] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:40:09 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 269 bytes in 74771 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:41:24.051548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:41:24.052256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:41:24.058876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:24.059365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:24.059805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:24.060497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:24.060954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:24.061383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:24.074169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:41:24.075699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:41:24.077284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:41:24.078326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:41:24.079833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:41:24.081201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:41:24.081201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc4d6d74-3b6d-4525-b5d0-d440c96ed5d3 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.146149 Sep 18 06:41:24.081201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1148/2200] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:41:23 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 111 bytes in 147 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:41:24.082046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:41:24.082610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:41:24.088004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:24.088480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:24.088914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:24.089589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:24.090043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:24.090472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:24.888876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [req-ece2caf3-951d-462d-bf2e-197cf1215e3e req-432753cf-aab4-451c-a9ae-ce75142b763e service nova] [30a3a4b1-8749-43a6-8164-1025dc424aa6] Reconnected to AMQP server on 10.222.0.20:5672 via [amqp] client with port 48444. Sep 18 06:41:24.892445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-ece2caf3-951d-462d-bf2e-197cf1215e3e req-432753cf-aab4-451c-a9ae-ce75142b763e service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 1.044870 Sep 18 06:41:24.893042 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1053/2201] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:41:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 1046 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:41:24.898497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:41:24.899209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:41:24.906430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:24.907494 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:24.908284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:24.909612 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:41:24.910398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:41:24.910876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:41:34.086258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:41:34.109179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=10 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:41:34.902548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=8 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:41:34.934020 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=18 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:42:09.287119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:42:09.372785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:42:19.291578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:42:19.378216 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:42:45.262295 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:42:45.262961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:42:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:43:00.001929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:43:00.002512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:43:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:43:05.341556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:43:05.342828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:43:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:43:06.399202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:43:06.399759 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:43:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:43:24.117916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7077a10-694b-496c-a819-6bff7809af98) Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:43:24.120159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:43:24.121826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7077a10-694b-496c-a819-6bff7809af98) Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:43:24.123083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7077a10-694b-496c-a819-6bff7809af98) Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:43:24.124657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7077a10-694b-496c-a819-6bff7809af98) Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.126448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7077a10-694b-496c-a819-6bff7809af98) Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:43:24.128351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7077a10-694b-496c-a819-6bff7809af98) Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7077a10-694b-496c-a819-6bff7809af98) Sep 18 06:43:24.129800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.131355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7077a10-694b-496c-a819-6bff7809af98) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:43:24.131355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-092a8df8-7bc5-457e-bac4-97be32416427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 120.043115 Sep 18 06:43:24.131355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1149/2202] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:41:24 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 269 bytes in 120044 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:43:24.131355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:43:24.131730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:43:24.137620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.138105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.138541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.139269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.139725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.140154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.153086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a9e70441-747c-4706-a80b-be12a9e7fce0) Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:43:24.155606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:43:24.157032 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a9e70441-747c-4706-a80b-be12a9e7fce0) Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:43:24.157859 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a9e70441-747c-4706-a80b-be12a9e7fce0) Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:43:24.158940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a9e70441-747c-4706-a80b-be12a9e7fce0) Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.160134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a9e70441-747c-4706-a80b-be12a9e7fce0) Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:43:24.161198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a9e70441-747c-4706-a80b-be12a9e7fce0) Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a9e70441-747c-4706-a80b-be12a9e7fce0) Sep 18 06:43:24.162438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.165229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a9e70441-747c-4706-a80b-be12a9e7fce0) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:43:24.165229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ac40126e-78c0-4a13-b674-ec3982452a9b tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 119.262810 Sep 18 06:43:24.165229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1054/2203] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:41:24 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 269 bytes in 119264 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:43:24.170510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-77ac3b5a-815d-4c2f-9db5-0105b617d3da tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047184 Sep 18 06:43:24.171525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1150/2204] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:43:24 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:43:24.176597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:43:24.177203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:43:24.182989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.183474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.183909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.184781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.185424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.185856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.200722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:43:24.202258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:43:24.203696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:43:24.205220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:43:24.207263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:43:24.209725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:43:24.209725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-86c68009-7ede-4291-85c1-64022ae2c78c tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.031303 Sep 18 06:43:24.209725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1151/2205] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:43:24 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:43:24.209725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:43:24.210419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:43:24.213563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:43:24.214179 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:43:24.215581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.216057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.216491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.217173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.217618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.218035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.220350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.221025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.221588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.223158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.223158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.223158 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.289407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:43:24.297079 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ebfc3434-6a82-4c03-b976-6d2f79c7dcc7 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1220 microversion: 2.1 time: 0.138272 Sep 18 06:43:24.297649 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1055/2206] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:43:24 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1220 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:43:24.302619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:43:24.303098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:43:24.308613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.309098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.309536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.310224 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.310700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.311151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.336947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120380) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:43:24.411305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6923fb90-28cf-4707-8cb4-be11747541bc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.112401 Sep 18 06:43:24.411883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1056/2207] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:43:24 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:43:24.417126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:43:24.417706 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:43:24.423314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.424557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.425078 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:24.425927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:43:24.426778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:43:24.427555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:43:34.212997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=6 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:43:34.238236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=12 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:43:34.421252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:43:34.449729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:44:09.438954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:44:09.485148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:44:19.443257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:44:19.489910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:44:28.481161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:44:28.481580 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:44:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:46:05.273030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:46:05.273535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:46:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:46:09.621748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-325e96ac-d139-4971-a0d8-0055d4c5f216) Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:46:09.623930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:46:09.625352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-325e96ac-d139-4971-a0d8-0055d4c5f216) Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:46:09.626068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-325e96ac-d139-4971-a0d8-0055d4c5f216) Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:46:09.628333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-325e96ac-d139-4971-a0d8-0055d4c5f216) Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.630258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-325e96ac-d139-4971-a0d8-0055d4c5f216) Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:46:09.631590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-325e96ac-d139-4971-a0d8-0055d4c5f216) Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-325e96ac-d139-4971-a0d8-0055d4c5f216) Sep 18 06:46:09.632995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.634569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-325e96ac-d139-4971-a0d8-0055d4c5f216) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:46:09.634569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-70b0d04f-33cb-483f-bbdd-6477a41779e8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 165.420260 Sep 18 06:46:09.634569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1152/2208] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:43:24 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 269 bytes in 165421 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:46:09.634569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:46:09.634966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:46:09.640843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.641324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.641753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.642431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.642903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.643321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.671810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c8380d34-f0ef-41c9-9f65-81e49c741ba8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044906 Sep 18 06:46:09.672563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1153/2209] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:46:09 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 06:46:09.677168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:46:09.677766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:46:09.683395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.683870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.684305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.684975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.685425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.685845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.707611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ac7a9ec2-6352-4cdd-827f-163433a1ef56) Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:46:09.709876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:46:09.711243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ac7a9ec2-6352-4cdd-827f-163433a1ef56) Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:46:09.712130 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ac7a9ec2-6352-4cdd-827f-163433a1ef56) Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:46:09.714391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ac7a9ec2-6352-4cdd-827f-163433a1ef56) Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.717036 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ac7a9ec2-6352-4cdd-827f-163433a1ef56) Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:46:09.719173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ac7a9ec2-6352-4cdd-827f-163433a1ef56) Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ac7a9ec2-6352-4cdd-827f-163433a1ef56) Sep 18 06:46:09.721532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.723539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ac7a9ec2-6352-4cdd-827f-163433a1ef56) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:46:09.723539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ac613220-2f2e-4ba0-916d-0531dce317f6 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 165.298493 Sep 18 06:46:09.723539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1057/2210] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:43:24 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 269 bytes in 165300 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 06:46:09.730028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:46:09.730701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:46:09.738884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.739369 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.739825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.740534 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.740986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.741408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.751033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:46:09.755039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:46:09.756718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:46:09.760012 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:46:09.761256 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:46:09.763331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:46:09.764803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:46:09.764803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-af74ff7c-9716-4335-a42c-5804446419c8 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.044850 Sep 18 06:46:09.765042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-05a6d858-c56a-4d16-a555-d11838382ac5 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1220 microversion: 2.1 time: 0.085320 Sep 18 06:46:09.765145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1058/2211] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:46:09 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:46:09.765242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1154/2212] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:46:09 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:46:09.765384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:46:09.765490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:46:09.765635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:46:09.765801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:46:09.769940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.770124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.770417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.770597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.770886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.771054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.771616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.771783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.772001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.772110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.772418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.772529 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.797455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:46:09.845217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:46:09.846651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8ac24cd9-cba1-4e65-92bf-2bb57115d076 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.085994 Sep 18 06:46:09.847307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1155/2213] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:46:09 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:46:09.853034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:46:09.853513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:46:09.859564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.860041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.860474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:09.861851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:46:09.862306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:46:09.862754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:46:19.767893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:46:19.790550 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:46:19.857296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=8 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:46:19.880560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=29,nova_cell1:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:46:20.010790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:46:20.010955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:46:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:46:25.352055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:46:25.352614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:46:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:46:26.401139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:46:26.401694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:46:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:46:26.408296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:46:26.408417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:46:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:47:48.492454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:47:48.492966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:47:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:48:09.784535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:48:09.824039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:48:19.788664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:48:19.835951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:49:25.284281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:49:25.284754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:49:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:49:40.020590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:49:40.021100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:49:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:49:45.362183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:49:45.362804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:49:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:49:46.411293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:49:46.411890 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:49:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:49:46.416135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:49:46.416288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:49:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:50:10.184852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f3378370-2d9c-4c71-9e3a-df1e1326d4e7) Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:50:10.187058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:50:10.188715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f3378370-2d9c-4c71-9e3a-df1e1326d4e7) Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:50:10.189660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f3378370-2d9c-4c71-9e3a-df1e1326d4e7) Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:50:10.190836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f3378370-2d9c-4c71-9e3a-df1e1326d4e7) Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.194177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f3378370-2d9c-4c71-9e3a-df1e1326d4e7) Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:50:10.195684 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f3378370-2d9c-4c71-9e3a-df1e1326d4e7) Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f3378370-2d9c-4c71-9e3a-df1e1326d4e7) Sep 18 06:50:10.197140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.198475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f3378370-2d9c-4c71-9e3a-df1e1326d4e7) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:50:10.198475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 240.428252 Sep 18 06:50:10.198475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:50:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:50:10.198475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:50:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (10.222.0.20) Sep 18 06:50:10.198475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-ba3298cf-25b5-428f-a095-152195205f84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Unhandled error: OSError: write error Sep 18 06:50:10.198475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:50:10.198475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:50:10.199175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1059/2214] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:46:09 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 0 bytes in 240430 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:50:10.199175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:50:10.199175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:50:10.204243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.204721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.205152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.205851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.206307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.206769 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.219455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5306314b-d5c6-43aa-9b60-95eb18be9bff) Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:50:10.222363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:50:10.223750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5306314b-d5c6-43aa-9b60-95eb18be9bff) Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:50:10.224561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5306314b-d5c6-43aa-9b60-95eb18be9bff) Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:50:10.225637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5306314b-d5c6-43aa-9b60-95eb18be9bff) Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.226910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5306314b-d5c6-43aa-9b60-95eb18be9bff) Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:50:10.227983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5306314b-d5c6-43aa-9b60-95eb18be9bff) Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5306314b-d5c6-43aa-9b60-95eb18be9bff) Sep 18 06:50:10.229100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.232300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5306314b-d5c6-43aa-9b60-95eb18be9bff) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:50:10.232300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 240.375469 Sep 18 06:50:10.232300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:50:10.232300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (10.222.0.20) Sep 18 06:50:10.232300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-842040a9-c250-415e-87ce-f343f7c5b13a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Unhandled error: OSError: write error Sep 18 06:50:10.232300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:50:10.232300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:50:10.233320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1156/2215] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:46:09 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 0 bytes in 240378 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:50:10.237702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046792 Sep 18 06:50:10.238213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:50:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments (ip 10.222.0.20) !!! Sep 18 06:50:10.238213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:50:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments (10.222.0.20) Sep 18 06:50:10.238837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-beec580f-bd8b-4eb7-a420-18f9d513c1cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Unhandled error: OSError: write error Sep 18 06:50:10.238837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:50:10.238837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:50:10.239094 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1060/2216] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:50:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 0 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:50:10.392040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:50:10.392652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:50:10.399057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.399707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.400300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.401144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.402272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.403197 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.417717 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:50:10.419267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:50:10.420637 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:50:10.421555 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:50:10.422824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:50:10.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:50:10.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.180849 Sep 18 06:50:10.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:50:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (ip 10.222.0.20) !!! Sep 18 06:50:10.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:50:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (10.222.0.20) Sep 18 06:50:10.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-40ac9284-29f6-45d2-985b-f0df9f43b28a tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Unhandled error: OSError: write error Sep 18 06:50:10.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:50:10.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:50:10.424396 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1061/2217] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:50:10 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 182 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:50:10.427173 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:50:10.427652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:50:10.433406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.433886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.434318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.435037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.435479 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.436107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.451786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:50:10.452671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:50:10.460117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.460601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.461068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.461760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.462212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.462660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.523830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:50:10.532203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1220 microversion: 2.1 time: 0.304431 Sep 18 06:50:10.532744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:50:10.532744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (10.222.0.20) Sep 18 06:50:10.533326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-2ad344fd-d11b-426c-a79e-84ef05906b4d tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:50:10.533326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:50:10.533326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:50:10.533590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1157/2218] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:50:10 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:50:10.537868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:50:10.538364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:50:10.544384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.544867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.545307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.545996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.546455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.546919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.570948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:50:10.617200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:50:10.617946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.083615 Sep 18 06:50:10.618453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (ip 10.222.0.20) !!! Sep 18 06:50:10.618453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (10.222.0.20) Sep 18 06:50:10.619058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-bc18bbbe-6fc3-48b9-bc29-d87471956dd2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Unhandled error: OSError: write error Sep 18 06:50:10.619058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:50:10.619058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:50:10.619322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1158/2219] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:50:10 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:50:10.750179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:50:10.750714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:50:10.756551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.757038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.757478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.758424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.758913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.759344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.786245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:50:10.866076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.245167 Sep 18 06:50:10.866678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (ip 10.222.0.20) !!! Sep 18 06:50:10.866678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (10.222.0.20) Sep 18 06:50:10.867612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-05f66369-53db-4785-a34e-7f507e16ac35 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Unhandled error: OSError: write error Sep 18 06:50:10.867612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:50:10.867612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:50:10.867612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1159/2220] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:50:10 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:50:10.926046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-6d16ab3f-fee4-4e01-b355-2bf7071de97d req-f2ddc29f-0e9f-4162-9cab-3d13162c3bda service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb5ac07d-bc0f-455d-a133-6abb2e30f695", "name": "network-vif-deleted", "tag": "72dd4c69-429a-42fe-8a60-ffe8316cc094"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:50:10.935306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6d16ab3f-fee4-4e01-b355-2bf7071de97d req-f2ddc29f-0e9f-4162-9cab-3d13162c3bda service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.935781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6d16ab3f-fee4-4e01-b355-2bf7071de97d req-f2ddc29f-0e9f-4162-9cab-3d13162c3bda service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.936209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6d16ab3f-fee4-4e01-b355-2bf7071de97d req-f2ddc29f-0e9f-4162-9cab-3d13162c3bda service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.946863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-6d16ab3f-fee4-4e01-b355-2bf7071de97d req-f2ddc29f-0e9f-4162-9cab-3d13162c3bda service nova] Creating event network-vif-deleted:72dd4c69-429a-42fe-8a60-ffe8316cc094 for instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 on hv-cinder-85535 Sep 18 06:50:10.952127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-6d16ab3f-fee4-4e01-b355-2bf7071de97d req-f2ddc29f-0e9f-4162-9cab-3d13162c3bda service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082675 Sep 18 06:50:10.952627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.20) !!! Sep 18 06:50:10.952627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:50:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/os-server-external-events (10.222.0.20) Sep 18 06:50:10.953280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [req-6d16ab3f-fee4-4e01-b355-2bf7071de97d req-f2ddc29f-0e9f-4162-9cab-3d13162c3bda service nova] Unhandled error: OSError: write error Sep 18 06:50:10.953280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:50:10.953280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:50:10.953548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1160/2221] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:50:10 2022] POST /compute/v2.1/os-server-external-events => generated 0 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:50:10.958230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:50:10.958846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:50:10.965103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.965582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.966015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:10.966720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:50:10.967175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:50:10.967599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:50:20.430954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:50:20.454111 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=12 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:50:20.962452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=15,nova_api:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:50:20.984610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=39,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:51:08.502076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:51:08.502818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:51:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:52:10.632500 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:52:10.723021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:52:20.636950 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:52:20.727400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:52:45.294753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:52:45.295208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:52:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:53:00.032989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:53:00.033571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:53:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:53:05.371809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:53:05.372458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:53:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:53:06.422723 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:53:06.423255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:53:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:53:06.425761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:53:06.425881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:53:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:54:10.935943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff297410-e83b-4a9a-8653-9803cd753cc9) Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:54:10.938107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:54:10.939867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff297410-e83b-4a9a-8653-9803cd753cc9) Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:54:10.940862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff297410-e83b-4a9a-8653-9803cd753cc9) Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:54:10.942966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff297410-e83b-4a9a-8653-9803cd753cc9) Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.944408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff297410-e83b-4a9a-8653-9803cd753cc9) Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:54:10.945658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff297410-e83b-4a9a-8653-9803cd753cc9) Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff297410-e83b-4a9a-8653-9803cd753cc9) Sep 18 06:54:10.948570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.949906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff297410-e83b-4a9a-8653-9803cd753cc9) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:54:10.949906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 240.516940 Sep 18 06:54:10.949906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:54:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:54:10.949906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:54:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (10.222.0.20) Sep 18 06:54:10.949906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-21c3ed3e-d05a-45f6-9eb6-cd9d99d2f4e6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Unhandled error: OSError: write error Sep 18 06:54:10.949906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:54:10.949906 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:54:10.950514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1062/2222] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:50:10 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 0 bytes in 240518 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:54:10.950514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:54:10.950514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:54:10.955537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:10.956048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:10.956486 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:10.957382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:10.957631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:10.958061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:10.975315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-24e6681c-f2ce-4d9e-810f-806270f92e35) Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:54:10.977632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:54:10.979120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-24e6681c-f2ce-4d9e-810f-806270f92e35) Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:54:10.979975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-24e6681c-f2ce-4d9e-810f-806270f92e35) Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:54:10.981140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-24e6681c-f2ce-4d9e-810f-806270f92e35) Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.982421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-24e6681c-f2ce-4d9e-810f-806270f92e35) Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:54:10.985283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-24e6681c-f2ce-4d9e-810f-806270f92e35) Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-24e6681c-f2ce-4d9e-810f-806270f92e35) Sep 18 06:54:10.988221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:54:10.990542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-24e6681c-f2ce-4d9e-810f-806270f92e35) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:54:10.990542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 240.024461 Sep 18 06:54:10.990542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:54:10.990542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (10.222.0.20) Sep 18 06:54:10.990542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-979a88bb-6827-4499-8d3f-58475b83c619 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Unhandled error: OSError: write error Sep 18 06:54:10.990542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:54:10.990542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:54:10.991695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1161/2223] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:50:10 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 0 bytes in 240026 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:54:10.991695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:54:10.991695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:54:10.992153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047387 Sep 18 06:54:10.992153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:54:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments (ip 10.222.0.20) !!! Sep 18 06:54:10.992153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:54:10 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments (10.222.0.20) Sep 18 06:54:10.992153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-7738cb43-dc07-4e51-9c51-189519f2b28a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Unhandled error: OSError: write error Sep 18 06:54:10.992153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:54:10.992153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:54:10.992153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1063/2224] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:54:10 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 0 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:54:10.995703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:54:10.995899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:10.996419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:10.996611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] [instance: 045c9108-b1cd-4720-94bc-7aa9159c26d4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:54:10.996788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:10.997518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:10.998537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:10.998841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.002808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.003301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.003735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.004432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.004883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.005314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.019399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] HTTP exception thrown: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Request method failure captured: Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 HTTP/1.1 Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmrTl7Q_802HLXOUje_MgoBFKX9opZX6bV38-5AXM8Wu6SS1UB3orQ4CCFBCsuiDtwSrbU_ZEtUu920re_YQNlYR0L5PpaqKYxuSK5IVsZNo1nWiAEDi3eh86kgHYcTHm6bRBZmQFKZkTZKRErD6_DcFuG1kgsjJVUpFDZaufr8Ihw Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:54:11.020944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9cbc859294e9456eb2ca751279a17f52"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 9cbc859294e9456eb2ca751279a17f52 Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeTestJSON-282001955 Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 6c945a3e6a4946559aeadc63566d29f4 Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeTestJSON-282001955-project Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:54:11.024135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '045c9108-b1cd-4720-94bc-7aa9159c26d4'} Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:54:11.026741 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. Sep 18 06:54:11.030368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:54:11.033132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Returning 404 to user: Instance 045c9108-b1cd-4720-94bc-7aa9159c26d4 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:54:11.033132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 404 len: 111 microversion: 2.1 time: 0.030344 Sep 18 06:54:11.033132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:54:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (ip 10.222.0.20) !!! Sep 18 06:54:11.033132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:54:11 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (10.222.0.20) Sep 18 06:54:11.033132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-3ad83d27-3361-417e-a68c-2659f0d7b947 tempest-AttachVolumeTestJSON-282001955 tempest-AttachVolumeTestJSON-282001955-project] Unhandled error: OSError: write error Sep 18 06:54:11.033132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:54:11.033132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:54:11.033132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1064/2225] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:54:10 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:54:11.034361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:54:11.034361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:54:11.035015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.035507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.035944 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.036639 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.037091 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.037566 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.064046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c2b511ca42e6427aa2f1627c87ad3fad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0adbb8b987449f08ee39d325424b1da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:54:11.072379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 200 len: 1220 microversion: 2.1 time: 0.090991 Sep 18 06:54:11.072905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:54:11.072905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:11 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (10.222.0.20) Sep 18 06:54:11.073490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-11177fd6-eb2c-4e33-8466-acea3766c7f9 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:54:11.073490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:54:11.073490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:54:11.073766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1162/2226] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:54:10 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:54:11.078547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:54:11.079110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:54:11.084687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.085169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.085608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.086294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.086793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.087223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.111746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:54:11.161082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:54:11.162041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.087474 Sep 18 06:54:11.162546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (ip 10.222.0.20) !!! Sep 18 06:54:11.162546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:11 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (10.222.0.20) Sep 18 06:54:11.163213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-1a3b0450-90ca-49c6-9bb0-bc069688d3f1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Unhandled error: OSError: write error Sep 18 06:54:11.163213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:54:11.163213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:54:11.163500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1163/2227] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:54:11 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:54:11.167712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:54:11.168198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:54:11.174170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.174686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.175122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.175806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.176268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.176693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.201455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:54:11.258539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:54:11.259666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.095259 Sep 18 06:54:11.260190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (ip 10.222.0.20) !!! Sep 18 06:54:11.260190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:11 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (10.222.0.20) Sep 18 06:54:11.260766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-ca08eeb5-1809-4e97-8247-607166c442ee tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Unhandled error: OSError: write error Sep 18 06:54:11.260766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:54:11.260766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:54:11.261067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1164/2228] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:54:11 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:54:11.265503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:54:11.265989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:54:11.275884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.277071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.277718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.278704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:54:11.279237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:54:11.279781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:54:11.666736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:54:11.718173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'services' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:54:21.035361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:54:21.269149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=8 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:54:21.671843 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=16 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:54:21.722734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:54:28.514131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:54:28.514679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:54:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:55:25.354162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03d61aee-f44d-4829-af8f-a506ebeeb01c) Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:55:25.356528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:55:25.358058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03d61aee-f44d-4829-af8f-a506ebeeb01c) Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:55:25.358779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03d61aee-f44d-4829-af8f-a506ebeeb01c) Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:55:25.359936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03d61aee-f44d-4829-af8f-a506ebeeb01c) Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.361946 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03d61aee-f44d-4829-af8f-a506ebeeb01c) Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:25.363810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03d61aee-f44d-4829-af8f-a506ebeeb01c) Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03d61aee-f44d-4829-af8f-a506ebeeb01c) Sep 18 06:55:25.365217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.366916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03d61aee-f44d-4829-af8f-a506ebeeb01c) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:25.366916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 74.333537 Sep 18 06:55:25.366916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:55:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:55:25.366916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:55:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (10.222.0.20) Sep 18 06:55:25.366916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-7e212dbf-04b2-4687-bc80-8b4a809ed697 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Unhandled error: OSError: write error Sep 18 06:55:25.366916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:55:25.366916 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:55:25.367664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1065/2229] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:54:11 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 0 bytes in 74335 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:25.367664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:25.367664 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:25.372988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.373472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.373907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.374603 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.375097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.375517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.404998 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044891 Sep 18 06:55:25.405495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:55:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments (ip 10.222.0.20) !!! Sep 18 06:55:25.405495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:55:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments (10.222.0.20) Sep 18 06:55:25.406656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-dd553ead-0035-4bfb-94cf-58dfc7645b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Unhandled error: OSError: write error Sep 18 06:55:25.406656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:55:25.406656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:55:25.406938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1066/2230] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:55:25 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 0 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:25.588564 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:25.589192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:25.595571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.596057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.596594 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.597261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.597711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.598181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.611563 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 HTTP/1.1 Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:55:25.613092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:25.614839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '4ce28d02-87b5-461d-bac3-2bf15b26fea9'} Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:25.615844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:25.617332 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.618747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:25.618747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 404 len: 111 microversion: 2.1 time: 0.206825 Sep 18 06:55:25.618747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:55:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:55:25.618747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:55:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (10.222.0.20) Sep 18 06:55:25.618747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:25.618747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:55:25.618747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:55:25.618747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1067/2231] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:25 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 208 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:25.669181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: WARNING keystonemiddleware.auth_token [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken: Token authorization failed Sep 18 06:55:25.670075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 401 len: 114 microversion: - time: 0.053593 Sep 18 06:55:25.670590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:55:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (ip 10.222.0.20) !!! Sep 18 06:55:25.670590 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:55:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (10.222.0.20) Sep 18 06:55:25.671137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-53bca0cf-8598-436e-a952-a6b71f2b8a27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:25.671137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:55:25.671137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:55:25.671380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1068/2232] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:25 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 55 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:25.675771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:25.676488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:25.682621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.683103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.683714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.684394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.684844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.685263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.735660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2cb34bc5-cbe2-4fce-b48f-2aebe3ae5c89) Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 HTTP/1.1 Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:25.741220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 06:55:25.743145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2cb34bc5-cbe2-4fce-b48f-2aebe3ae5c89) Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'server_id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e', 'id': '466d74bb-d7b4-47ff-a708-ab67d2eec121'} Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:55:25.743952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2cb34bc5-cbe2-4fce-b48f-2aebe3ae5c89) Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:55:25.745092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2cb34bc5-cbe2-4fce-b48f-2aebe3ae5c89) Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.750857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2cb34bc5-cbe2-4fce-b48f-2aebe3ae5c89) Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:25.753628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2cb34bc5-cbe2-4fce-b48f-2aebe3ae5c89) Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2cb34bc5-cbe2-4fce-b48f-2aebe3ae5c89) Sep 18 06:55:25.758290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:25.761934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2cb34bc5-cbe2-4fce-b48f-2aebe3ae5c89) {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:25.761934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121" status: 400 len: 269 microversion: 2.1 time: 74.481051 Sep 18 06:55:25.761934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (ip 10.222.0.20) !!! Sep 18 06:55:25.761934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 (10.222.0.20) Sep 18 06:55:25.761934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-5aa0389c-6386-446a-ae87-f3878e0db81a tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Unhandled error: OSError: write error Sep 18 06:55:25.761934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:25.761934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:25.765464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1165/2233] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:54:11 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e/os-volume_attachments/466d74bb-d7b4-47ff-a708-ab67d2eec121 => generated 0 bytes in 74483 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:25.765464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:25.765464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:25.765464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.767260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.767260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.767260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.767260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.767820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.798626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:25.846739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:55:25.847562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.102240 Sep 18 06:55:25.848084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (ip 10.222.0.20) !!! Sep 18 06:55:25.848084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:25 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (10.222.0.20) Sep 18 06:55:25.848677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-8ef41161-ab2c-4445-8da4-5b77a64ba426 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Unhandled error: OSError: write error Sep 18 06:55:25.848677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:25.848677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:25.848949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1166/2234] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:25 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:25.960169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:25.960677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:25.966564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.967076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.967519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.968207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:25.968687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:25.969114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:25.999522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.046417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.047752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.198074 Sep 18 06:55:26.048257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (ip 10.222.0.20) !!! Sep 18 06:55:26.048257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (10.222.0.20) Sep 18 06:55:26.048833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-9d1eb375-93a0-4b2c-acb8-0396d5e68840 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Unhandled error: OSError: write error Sep 18 06:55:26.048833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.048833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.049099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1167/2235] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:25 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.053736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.055999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.065019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.065506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.065945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.066677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.067131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.067561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.099284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049327 Sep 18 06:55:26.099796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments (ip 10.222.0.20) !!! Sep 18 06:55:26.099796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments (10.222.0.20) Sep 18 06:55:26.100368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-8c010a88-a72e-4779-8682-5c6c2a1385c6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Unhandled error: OSError: write error Sep 18 06:55:26.100368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.100368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.100735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1168/2236] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16/os-volume_attachments => generated 0 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.104711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.105300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.111249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.111732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.112162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.112838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.113289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.113748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.130306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 HTTP/1.1 Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:26.131841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '4ce28d02-87b5-461d-bac3-2bf15b26fea9'} Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:26.134413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.135999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 404 len: 111 microversion: 2.1 time: 0.031837 Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (10.222.0.20) Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1169/2237] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.137467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Sep 18 06:55:26.138239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 401 len: 114 microversion: - time: 0.001903 Sep 18 06:55:26.138239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (ip 10.222.0.20) !!! Sep 18 06:55:26.138239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (10.222.0.20) Sep 18 06:55:26.138239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-92d20b6d-3db0-48f8-843d-851e9f3bcbf8 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.138239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.138239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.138239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1170/2238] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.142190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.142695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.148037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.148511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.148942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.149611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.150062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.150484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.175254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.218023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.218811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.079910 Sep 18 06:55:26.219309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (ip 10.222.0.20) !!! Sep 18 06:55:26.219309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (10.222.0.20) Sep 18 06:55:26.219879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-e8772b57-1ca2-4ea3-a390-f09cfcc6e160 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Unhandled error: OSError: write error Sep 18 06:55:26.219879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.219879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.220140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1171/2239] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.224266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.224749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.230061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.230541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.231008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.231687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.232140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.232564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.257430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.299885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.300681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.079807 Sep 18 06:55:26.301173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (ip 10.222.0.20) !!! Sep 18 06:55:26.301173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (10.222.0.20) Sep 18 06:55:26.301743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-a1d164fc-03a2-44e6-867b-95eb3d8eaa98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Unhandled error: OSError: write error Sep 18 06:55:26.301743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.301743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.302015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1172/2240] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.306150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.306781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.311986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.313101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.313541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.314214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.314700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.315126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.327682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 HTTP/1.1 Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:26.329210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.330921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '4ce28d02-87b5-461d-bac3-2bf15b26fea9'} Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:26.331832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.333247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 404 len: 111 microversion: 2.1 time: 0.027958 Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (10.222.0.20) Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1173/2241] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.334488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Sep 18 06:55:26.335715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 401 len: 114 microversion: - time: 0.001872 Sep 18 06:55:26.335715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (ip 10.222.0.20) !!! Sep 18 06:55:26.335715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (10.222.0.20) Sep 18 06:55:26.335715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-f326810e-e173-4293-8c2f-57f6689ebaea tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.335715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.335715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.335715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1174/2242] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.340080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.340571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.346206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.346730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.347168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.347858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.348296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.348718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.375340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.419473 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.420285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.083899 Sep 18 06:55:26.420785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (ip 10.222.0.20) !!! Sep 18 06:55:26.420785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (10.222.0.20) Sep 18 06:55:26.421358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-aeae97b4-3e05-4802-b0e0-2c43961b6ef4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Unhandled error: OSError: write error Sep 18 06:55:26.421358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.421358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.422301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1175/2243] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.426340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.426851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.432404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.432884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.433324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.434003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.434456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.434906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.460644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.505317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.506064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.083138 Sep 18 06:55:26.506592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (ip 10.222.0.20) !!! Sep 18 06:55:26.506592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (10.222.0.20) Sep 18 06:55:26.507171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-35e18672-9b57-4f9b-a49a-55fb522e8c63 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Unhandled error: OSError: write error Sep 18 06:55:26.507171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.507171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.507465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1176/2244] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.511623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.512222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.517564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.518042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.518481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.519195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.519654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.520080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.533267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 HTTP/1.1 Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:26.534789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.536380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '4ce28d02-87b5-461d-bac3-2bf15b26fea9'} Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:26.537252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.538846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 404 len: 111 microversion: 2.1 time: 0.027974 Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (10.222.0.20) Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1177/2245] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.540286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Sep 18 06:55:26.541125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 401 len: 114 microversion: - time: 0.002272 Sep 18 06:55:26.541125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (ip 10.222.0.20) !!! Sep 18 06:55:26.541125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (10.222.0.20) Sep 18 06:55:26.541370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-2f724566-ac81-42ca-aba1-ef7a5a8f5c27 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.541370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.541370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.541611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1178/2246] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 4 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.545539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.546027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.552025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.552502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.552939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.553613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.554066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.554490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.572235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=6 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:26.580660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.630392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.631237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.088964 Sep 18 06:55:26.631738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (ip 10.222.0.20) !!! Sep 18 06:55:26.631738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (10.222.0.20) Sep 18 06:55:26.632312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-d6716ea2-6be2-4000-beca-c377cc918169 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Unhandled error: OSError: write error Sep 18 06:55:26.632312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.632312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.632588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1179/2247] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.636803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.637291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.642697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.643184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.643625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.644313 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.644773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.645199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.669651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.714895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.715696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.082394 Sep 18 06:55:26.716192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (ip 10.222.0.20) !!! Sep 18 06:55:26.716192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (10.222.0.20) Sep 18 06:55:26.716760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-ae8253d2-a50c-48f6-aa92-b67b10724a11 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Unhandled error: OSError: write error Sep 18 06:55:26.716760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.716760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.717047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1180/2248] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.721604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.722200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.727796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.728277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.728711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.729382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.729827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.730243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.743344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 HTTP/1.1 Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:26.744810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.746443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '4ce28d02-87b5-461d-bac3-2bf15b26fea9'} Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:26.747414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.749034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 404 len: 111 microversion: 2.1 time: 0.028402 Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (10.222.0.20) Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1181/2249] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.750404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Sep 18 06:55:26.751178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 401 len: 114 microversion: - time: 0.001840 Sep 18 06:55:26.751178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (ip 10.222.0.20) !!! Sep 18 06:55:26.751178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (10.222.0.20) Sep 18 06:55:26.751178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-c4b59944-473e-4cff-8b4c-3554a17a2712 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.751178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.751178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.751178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1182/2250] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.755269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.755794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.760927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.761395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.761825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.762484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.762954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.763373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.787194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.831745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.832556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.080736 Sep 18 06:55:26.833040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (ip 10.222.0.20) !!! Sep 18 06:55:26.833040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (10.222.0.20) Sep 18 06:55:26.833605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-122b1ba1-52c9-4c8d-a558-4275e3a35ebc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Unhandled error: OSError: write error Sep 18 06:55:26.833605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.833605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.833877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1183/2251] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.837999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.838475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.844287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.844759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.845190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.845857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.846302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.846754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.870557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:26.915039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:26.915836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.081263 Sep 18 06:55:26.916329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (ip 10.222.0.20) !!! Sep 18 06:55:26.916329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (10.222.0.20) Sep 18 06:55:26.916897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-d1b56b8b-176d-4954-ae3b-4b2d21435810 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Unhandled error: OSError: write error Sep 18 06:55:26.916897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.916897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.917217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1184/2252] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.921276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.921898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] [instance: 4ce28d02-87b5-461d-bac3-2bf15b26fea9] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.927214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.927598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.928054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.928747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.929196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.929617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.942193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 HTTP/1.1 Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:26.943722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.945317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '4ce28d02-87b5-461d-bac3-2bf15b26fea9'} Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:26.946220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. Sep 18 06:55:26.947781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Instance 4ce28d02-87b5-461d-bac3-2bf15b26fea9 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9" status: 404 len: 111 microversion: 2.1 time: 0.027184 Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (ip 10.222.0.20) !!! Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 (10.222.0.20) Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1185/2253] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/4ce28d02-87b5-461d-bac3-2bf15b26fea9 => generated 0 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.949165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken Sep 18 06:55:26.949920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4" status: 401 len: 114 microversion: - time: 0.001875 Sep 18 06:55:26.949920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (ip 10.222.0.20) !!! Sep 18 06:55:26.949920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 (10.222.0.20) Sep 18 06:55:26.949920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-432f14f4-c2aa-4187-ae3c-b7ab42e5517e tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Unhandled error: OSError: write error Sep 18 06:55:26.949920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:26.949920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:26.949920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1186/2254] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:26 2022] GET /compute/v2.1/servers/045c9108-b1cd-4720-94bc-7aa9159c26d4 => generated 0 bytes in 3 msecs (HTTP/1.1 401) 6 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:26.954397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:26.954909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:26.960165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.960635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.961067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.961730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:26.962171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:26.962634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:26.986537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:27.028512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:55:27.029222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.078619 Sep 18 06:55:27.029707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (ip 10.222.0.20) !!! Sep 18 06:55:27.029707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:27 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 (10.222.0.20) Sep 18 06:55:27.030265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-2dbf1aa0-04d5-484a-b5a3-05e713b6ef7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Unhandled error: OSError: write error Sep 18 06:55:27.030265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:27.030265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:27.030538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1187/2255] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:26 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:27.035207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.035686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:27.040981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.041460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.041897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.042568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.043040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.043464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.066703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:27.108901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:27.109703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.078428 Sep 18 06:55:27.110265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (ip 10.222.0.20) !!! Sep 18 06:55:27.110265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 06:55:27 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e (10.222.0.20) Sep 18 06:55:27.110855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: CRITICAL nova [None req-23eb8caf-81e5-4b7f-a868-8c40cb0cf7f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Unhandled error: OSError: write error Sep 18 06:55:27.110855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova OSError: write error Sep 18 06:55:27.110855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova Sep 18 06:55:27.111132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1188/2256] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:55:27.115221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a60ad53a-4d86-4598-bb0d-b37895035468 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '.version_select at 0x7fe673c8eee0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.134770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a60ad53a-4d86-4598-bb0d-b37895035468 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792" status: 202 len: 0 microversion: 2.1 time: 0.022785 Sep 18 06:55:27.135301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1189/2257] 10.222.0.20 () {58 vars in 1254 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:55:27.179669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: WARNING keystonemiddleware.auth_token [None req-a60ad53a-4d86-4598-bb0d-b37895035468 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken: Token authorization failed Sep 18 06:55:27.180438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a60ad53a-4d86-4598-bb0d-b37895035468 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 401 len: 114 microversion: - time: 0.044259 Sep 18 06:55:27.180959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1190/2258] 10.222.0.20 () {60 vars in 1155 bytes} [Sun Sep 18 06:55:27 2022] POST /compute/v2.1/os-volumes => generated 114 bytes in 45 msecs (HTTP/1.1 401) 6 headers in 293 bytes (1 switches on core 0) Sep 18 06:55:27.185456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.185943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:27.193076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.193556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.193985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.194694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.195141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.195560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.221694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:27.264297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Instance is already in deleting state, ignoring this request Sep 18 06:55:27.265092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-94e113b7-1831-4677-a36c-34d64fcf2983 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 204 len: 0 microversion: 2.1 time: 0.083239 Sep 18 06:55:27.266271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1191/2259] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:55:27.270485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.270984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:27.276229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.276710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.277145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.277817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.278264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.278770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.282462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=7 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:27.303259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:27.346726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:27.347537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b4f60175-2321-4163-bba9-468641198542 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.080359 Sep 18 06:55:27.348189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1192/2260] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:55:27.352535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b7f994e5-f079-464c-8a1a-6175022b0347 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '.version_select at 0x7fe6703ca940>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.367662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-b7f994e5-f079-464c-8a1a-6175022b0347 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b7f994e5-f079-464c-8a1a-6175022b0347 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 HTTP/1.1 Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:27.371028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'tempest-TestShelveInstance-1890991792'} Sep 18 06:55:27.372678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise self.value Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 18 06:55:27.373671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:55:27.375757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.377285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.378838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b7f994e5-f079-464c-8a1a-6175022b0347 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:27.378838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b7f994e5-f079-464c-8a1a-6175022b0347 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792" status: 404 len: 143 microversion: 2.1 time: 0.023304 Sep 18 06:55:27.378838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1193/2261] 10.222.0.20 () {58 vars in 1254 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 => generated 143 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:55:27.382119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dd1942ad-140e-4fff-a784-e56103e2d796 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '.version_select at 0x7fe6701a75e0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.400778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dd1942ad-140e-4fff-a784-e56103e2d796 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041" status: 202 len: 0 microversion: 2.1 time: 0.026985 Sep 18 06:55:27.401332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1194/2262] 10.222.0.20 () {58 vars in 1215 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:55:27.405837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.406460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:27.411987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.412472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.412913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.413597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.414050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.414477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.438704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:27.482485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:27.483336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-746cd9e2-591a-4e63-87ff-14d8fee36490 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.081111 Sep 18 06:55:27.483894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1195/2263] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:55:27.488261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eaebf807-f2c9-473f-aa23-e4966ca42d01 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '.version_select at 0x7fe670752f70>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.499158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-eaebf807-f2c9-473f-aa23-e4966ca42d01 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eaebf807-f2c9-473f-aa23-e4966ca42d01 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 HTTP/1.1 Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:27.501417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'tempest-TestShelveInstance-1890991792'} Sep 18 06:55:27.503082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise self.value Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 18 06:55:27.504232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:55:27.506017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.507534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.508815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eaebf807-f2c9-473f-aa23-e4966ca42d01 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:27.508815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eaebf807-f2c9-473f-aa23-e4966ca42d01 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792" status: 404 len: 143 microversion: 2.1 time: 0.017961 Sep 18 06:55:27.508815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1196/2264] 10.222.0.20 () {58 vars in 1254 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 => generated 143 bytes in 19 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:55:27.508815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0cc2465e-2f30-4775-b84a-82b04a82b57f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '.version_select at 0x7fe673c4f940>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.519175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-0cc2465e-2f30-4775-b84a-82b04a82b57f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0cc2465e-2f30-4775-b84a-82b04a82b57f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041 HTTP/1.1 Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:27.520833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'tempest-keypair-30015041'} Sep 18 06:55:27.522551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise self.value Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 18 06:55:27.523855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:55:27.525278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.526833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.528131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0cc2465e-2f30-4775-b84a-82b04a82b57f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:27.528131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0cc2465e-2f30-4775-b84a-82b04a82b57f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041" status: 404 len: 130 microversion: 2.1 time: 0.018047 Sep 18 06:55:27.528131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1197/2265] 10.222.0.20 () {58 vars in 1215 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041 => generated 130 bytes in 19 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:55:27.528131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.528131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:27.532584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.533005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.533435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.534110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.534557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.535003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.559635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:27.605891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Instance is already in deleting state, ignoring this request Sep 18 06:55:27.606720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-66050e3d-a503-4522-880d-cbc2cc215472 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 204 len: 0 microversion: 2.1 time: 0.083255 Sep 18 06:55:27.607264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1198/2266] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:55:27.612060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e401d2a1-7761-47f7-b9e8-90c8c990f77a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '.version_select at 0x7fe6701a7e50>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.622487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-e401d2a1-7761-47f7-b9e8-90c8c990f77a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e401d2a1-7761-47f7-b9e8-90c8c990f77a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 HTTP/1.1 Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:27.624213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'tempest-TestShelveInstance-1890991792'} Sep 18 06:55:27.626627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise self.value Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 18 06:55:27.627704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:55:27.629214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.631044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.632330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e401d2a1-7761-47f7-b9e8-90c8c990f77a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:27.632330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e401d2a1-7761-47f7-b9e8-90c8c990f77a tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792" status: 404 len: 143 microversion: 2.1 time: 0.017997 Sep 18 06:55:27.632330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1199/2267] 10.222.0.20 () {58 vars in 1254 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 => generated 143 bytes in 19 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:55:27.632330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.634974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:27.641540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.641540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.641540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.642081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.642527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.642979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.667298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:27.741279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f72e5179-03ee-449c-b92c-d21eed12e352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 204 len: 0 microversion: 2.1 time: 0.113031 Sep 18 06:55:27.742063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1200/2268] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:55:27.747526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4a6286aa-adb3-4a04-ae1e-3c92f42056e9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '.version_select at 0x7fe6703d1820>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.759405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-4a6286aa-adb3-4a04-ae1e-3c92f42056e9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4a6286aa-adb3-4a04-ae1e-3c92f42056e9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041 HTTP/1.1 Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:27.761211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'tempest-keypair-30015041'} Sep 18 06:55:27.762858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise self.value Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 18 06:55:27.763918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:55:27.765397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:27.766871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.768149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4a6286aa-adb3-4a04-ae1e-3c92f42056e9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:27.768149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4a6286aa-adb3-4a04-ae1e-3c92f42056e9 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041" status: 404 len: 130 microversion: 2.1 time: 0.019307 Sep 18 06:55:27.768149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1201/2269] 10.222.0.20 () {58 vars in 1215 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041 => generated 130 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:55:27.768149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.768775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:27.774247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.774768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.775208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.775900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.776384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.776810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.901890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:27.910675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1598d2ab-8edc-4909-a097-312d50e481b8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146680 Sep 18 06:55:27.911230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1202/2270] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:27.915537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5d4bae73-1338-47b9-968e-b71007dc67b1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Calling method '.version_select at 0x7fe6704c8ee0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.927021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-5d4bae73-1338-47b9-968e-b71007dc67b1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] HTTP exception thrown: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5d4bae73-1338-47b9-968e-b71007dc67b1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Request method failure captured: Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 HTTP/1.1 Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJroh-CWrf7kmgYd-Ng0aukErIZfhp525BZjCBHZBmC26saWwbss1mIM7kUzk9MrCE-LPw7R7wx7SZUNiXQegl7IwSqHGtQbvPinNieCIb5tE-YdDHwtuucdifMT9YMewdE25MohCZAPv1gMtCW5JxJ6NulSgOdIFH0VscToGS59OYRY Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:27.928758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d0adbb8b987449f08ee39d325424b1da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: d0adbb8b987449f08ee39d325424b1da Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestShelveInstance-1214129446 Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestShelveInstance-1214129446-project Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'tempest-TestShelveInstance-1890991792'} Sep 18 06:55:27.930352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise self.value Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 18 06:55:27.931256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:55:27.932480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad Sep 18 06:55:27.933887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:27.934970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5d4bae73-1338-47b9-968e-b71007dc67b1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] Returning 404 to user: Keypair tempest-TestShelveInstance-1890991792 not found for user c2b511ca42e6427aa2f1627c87ad3fad {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:27.934970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5d4bae73-1338-47b9-968e-b71007dc67b1 tempest-TestShelveInstance-1214129446 tempest-TestShelveInstance-1214129446-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792" status: 404 len: 143 microversion: 2.1 time: 0.017970 Sep 18 06:55:27.934970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1203/2271] 10.222.0.20 () {58 vars in 1254 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1890991792 => generated 143 bytes in 19 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:55:27.934970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:27.935374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:27.942379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.942904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.943347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.944040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:27.944496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:27.944925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:27.969543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:55:28.016233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Instance is already in deleting state, ignoring this request Sep 18 06:55:28.017171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1f954664-973c-4b95-a464-48f964e25cc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 204 len: 0 microversion: 2.1 time: 0.085706 Sep 18 06:55:28.017993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1204/2272] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:55:27 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:55:28.022766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-90739fbf-142f-4c04-9eea-0c34b0d7e2a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '.version_select at 0x7fe673c4f430>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:28.035225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-90739fbf-142f-4c04-9eea-0c34b0d7e2a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-90739fbf-142f-4c04-9eea-0c34b0d7e2a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041 HTTP/1.1 Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:55:28.036987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'tempest-keypair-30015041'} Sep 18 06:55:28.038667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise self.value Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 18 06:55:28.039564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 06:55:28.040775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:55:28.042023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:55:28.043519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-90739fbf-142f-4c04-9eea-0c34b0d7e2a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Keypair tempest-keypair-30015041 not found for user d85160b01d824bc3a32b425fab98d2c7 {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:55:28.043519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-90739fbf-142f-4c04-9eea-0c34b0d7e2a4 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041" status: 404 len: 130 microversion: 2.1 time: 0.019544 Sep 18 06:55:28.043519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1205/2273] 10.222.0.20 () {58 vars in 1215 bytes} [Sun Sep 18 06:55:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-30015041 => generated 130 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:55:28.043519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:28.044896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:28.051693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:28.052185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:28.053046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:28.053319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:28.053775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:28.054205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:28.326265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:28.335761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8f28fa8c-ec68-4b6d-bad3-0cc687f7c4ec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.295848 Sep 18 06:55:28.336823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1206/2274] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:28 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:28.341502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:28.342108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:28.348532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:28.349279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:28.349721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:28.350407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:28.350887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:28.351388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:28.633078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:28.642470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eef81296-5ee3-458f-874d-e07e0304eb14 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.304671 Sep 18 06:55:28.643470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1207/2275] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:28 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:28.947376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a63018eb-2127-4d64-813c-590048967e6a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: .version_select at 0x7fe67057ae50>, body: {"keypair": {"name": "tempest-keypair-285570834", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBMSCsXB/dJ46KGstyInoi0SjdfoTndjDofMvbIQzxn8fATCyO3yeCHQImxA0LTQscjS+dKMlB4gR3I67zoUcqK1Sot5XNkXd7Teo75NXZUTr4p3h4NVb6qQZc6saHAUtw=="}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:55:28.965277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-a63018eb-2127-4d64-813c-590048967e6a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:55:28.969889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-a63018eb-2127-4d64-813c-590048967e6a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for user d85160b01d824bc3a32b425fab98d2c7 and project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:55:28.992955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a63018eb-2127-4d64-813c-590048967e6a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.050286 Sep 18 06:55:28.993838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1208/2276] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:55:28 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:55:29.372806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:29.373429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:29.396114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:29.396484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:29.396920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:29.397622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:29.398072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:29.402616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:29.648796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:29.661300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b9347a36-241c-48eb-b9b3-b2e14a6a6c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.303609 Sep 18 06:55:29.661984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1209/2277] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:29.668773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:29.669389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:29.681772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:29.682268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:29.682755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:29.683511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:29.683970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:29.684748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:29.922931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:29.926570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=5,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:29.932465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fee5b73a-8130-4b4d-b16a-461283de86ea tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.269356 Sep 18 06:55:29.939758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1210/2278] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:29 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:30.332133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-818817438", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d572beb8-e9a5-4e8d-a0f6-5ec49c88e496"}], "security_groups": [{"name": "tempest-securitygroup--1167730359"}], "key_name": "tempest-keypair-285570834", "user_data": "}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:55:30.344406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:30.478390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] validate_networks() for [('d572beb8-e9a5-4e8d-a0f6-5ec49c88e496', None, None, None, None, None)] {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:55:31.501383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:55:31.502446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:55:31.503161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:55:31.503934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:55:31.504931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:55:31.514914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:31.516161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:31.516773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:31.517694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:31.518325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:31.518945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:31.543106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:55:31.554037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for user d85160b01d824bc3a32b425fab98d2c7 and project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:55:31.566272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:55:31.569035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:55:31.624042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-714a8fb0-970c-499b-8640-4d80af355dd3 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.298508 Sep 18 06:55:31.625295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1211/2279] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:55:30 2022] POST /compute/v2.1/servers => generated 402 bytes in 1300 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:55:31.630286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:31.631099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:31.639098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:31.639846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:31.640865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:31.641828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:31.642295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:31.642762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:31.802190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:31.814901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-44c8ddba-2eee-4019-926f-605244817a22 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.188261 Sep 18 06:55:31.815794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1212/2280] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:31.828757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:31.829905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:31.837004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:31.837563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:31.838066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:31.838856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:31.850769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:31.851423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:32.067819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:32.076871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eeea841f-1b5d-4a62-86b2-0c77483ccef1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.260085 Sep 18 06:55:32.078085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1213/2281] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:31 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:32.083991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ea9b1071-0398-4d2d-a548-8e540c6441e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:32.084881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea9b1071-0398-4d2d-a548-8e540c6441e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:32.157163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ea9b1071-0398-4d2d-a548-8e540c6441e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:32.170178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ea9b1071-0398-4d2d-a548-8e540c6441e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1240 microversion: 2.1 time: 0.090769 Sep 18 06:55:32.171947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1214/2282] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:32 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1240 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:32.838687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:32.840098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:32.848936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:32.849424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:32.849993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:32.850717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:32.851173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:32.851596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:33.058451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:33.066796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b6875082-a0fe-4f04-be32-16fba649f708 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.231940 Sep 18 06:55:33.067586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1215/2283] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:32 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:33.104998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:33.105638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:33.111569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:33.112053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:33.112816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:33.113164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:33.113622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:33.114042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:33.422913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:33.433532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4fa2bfaf-1ad7-4d49-b747-88a1adc2f432 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.333784 Sep 18 06:55:33.434697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1216/2284] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:33 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:33.440925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:33.441535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:33.448136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:33.448753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:33.449259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:33.450702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:33.451358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:33.451781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:33.538676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:33.548741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0e2522b-394a-4bd2-a70d-3f63e8420452 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1298 microversion: 2.1 time: 0.111803 Sep 18 06:55:33.549699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1217/2285] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:33 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1298 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:34.090350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:34.091096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:34.097980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:34.098462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:34.098927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:34.099617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:34.100095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:34.100522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:34.383993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:34.392803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1a2128c2-07b2-406f-b709-4614a55a623e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.306308 Sep 18 06:55:34.393526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1218/2286] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:34.459476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:34.460335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:34.470541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:34.471065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:34.471500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:34.472194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:34.472644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:34.473064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:34.714220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:34.727351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3d4c34a4-ead8-4e17-b42a-be0a98fa6355 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.272676 Sep 18 06:55:34.732488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1219/2287] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:34 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:34.746501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:34.747196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:34.754988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:34.755736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:34.756381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:34.757334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:34.757956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:34.758640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:35.026621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:35.034945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-13f26474-ca10-4d2e-a866-e63258aad30d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1366 microversion: 2.1 time: 0.298447 Sep 18 06:55:35.037620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1220/2288] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:34 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1366 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:35.417311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:35.418203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:35.425642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:35.426258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:35.426946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:35.428012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:35.428640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:35.429691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:35.663407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:35.679953 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:35.686920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fe799823-7187-4648-af69-fc8d0081066a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.265915 Sep 18 06:55:35.687573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1221/2289] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:35.707138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=12 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:35.813931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-1eb41ba6-b97e-42f8-af01-2db0426f2110 req-ba5a4d2e-24a0-4767-9b71-312fb6dea62e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ceaad6c7-d3bd-45ed-9e68-010c68b41f1b", "tag": "0a70bc98-af92-4192-933a-5024fbdfa5f0"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:55:35.824912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1eb41ba6-b97e-42f8-af01-2db0426f2110 req-ba5a4d2e-24a0-4767-9b71-312fb6dea62e service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:35.825524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1eb41ba6-b97e-42f8-af01-2db0426f2110 req-ba5a4d2e-24a0-4767-9b71-312fb6dea62e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:35.826122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1eb41ba6-b97e-42f8-af01-2db0426f2110 req-ba5a4d2e-24a0-4767-9b71-312fb6dea62e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:35.841229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-1eb41ba6-b97e-42f8-af01-2db0426f2110 req-ba5a4d2e-24a0-4767-9b71-312fb6dea62e service nova] Creating event network-changed:0a70bc98-af92-4192-933a-5024fbdfa5f0 for instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b on hv-cinder-85535 Sep 18 06:55:35.846981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-1eb41ba6-b97e-42f8-af01-2db0426f2110 req-ba5a4d2e-24a0-4767-9b71-312fb6dea62e service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.114089 Sep 18 06:55:35.847857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1222/2290] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:55:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:55:35.853433 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:35.854394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:35.863516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:35.864017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:35.864549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:35.865286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:35.865747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:35.866188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:36.185245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:36.196650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bde03f7e-59fe-4cee-991f-08b1e827d5ba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.346669 Sep 18 06:55:36.211442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1223/2291] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:35 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:36.217498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:36.218359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:36.225150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:36.225639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:36.226075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:36.226813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:36.227451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:36.227881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:36.837962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:36.846509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-85921f2f-7ac8-427b-b0bc-a15e870859c2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1366 microversion: 2.1 time: 0.633681 Sep 18 06:55:36.847116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1224/2292] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:36 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1366 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:36.851499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:36.852213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:36.857847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:36.858324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:36.859302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:36.859971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:36.860418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:36.860836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:37.091494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:37.100627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b0ef3c67-c245-485d-b1db-5e61c3166bd0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.252424 Sep 18 06:55:37.101752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1225/2293] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:37.222759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:37.223377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:37.229553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:37.230041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:37.230607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:37.231374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:37.231777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:37.232204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:37.383415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:37.392780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-570908ef-1a7b-4eb9-8dee-d0bf0765c513 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.174490 Sep 18 06:55:37.393360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1226/2294] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:37 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:37.871487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:37.872195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:37.879528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:37.880090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:37.880630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:37.881362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:37.881816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:37.882242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:38.120263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:38.130344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8b015bbf-1674-4f7a-86b7-8f3fe36afd7f tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1366 microversion: 2.1 time: 0.262716 Sep 18 06:55:38.131168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1227/2295] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:37 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1366 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:38.135830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:38.136514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:38.142592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:38.143165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:38.143951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:38.145012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:38.145759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:38.146193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:38.294442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:38.304184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-85b71c92-aed7-48f1-a1f5-ebbbbe110eec tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.171771 Sep 18 06:55:38.304786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1228/2296] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:38 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:38.416837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:38.417489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:38.424488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:38.424979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:38.425425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:38.426140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:38.426612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:38.427040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:38.553466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:38.557010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:38.562611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fe8646a1-2441-439e-90f9-642ba410c594 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149670 Sep 18 06:55:38.563189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1229/2297] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:38 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:39.153276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:39.153916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:39.161097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:39.161573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:39.162006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:39.162733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:39.163235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:39.163659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:39.294456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:39.302028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8ebf91f2-4f2a-4a83-9d81-622059ff790d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1366 microversion: 2.1 time: 0.152940 Sep 18 06:55:39.302598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1230/2298] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:39 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:39.328456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:39.329045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:39.334482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:39.334992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:39.335422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:39.336095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:39.336544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:39.336963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:39.463492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:39.471849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8caceecb-bfc5-436d-9fa6-46ebb937242a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147554 Sep 18 06:55:39.472412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1231/2299] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:39 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:39.585159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:39.585810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:39.591622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:39.592137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:39.592574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:39.593296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:39.593748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:39.594171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:39.718193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:39.726268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2cfbba7e-a63a-49c9-9e62-6b2bbafbd84f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.144738 Sep 18 06:55:39.726860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1232/2300] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:39 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:40.325264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:40.325977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:40.332741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:40.333217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:40.333642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:40.334314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:40.334794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:40.335213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:40.463995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:40.472310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-10db7052-f2b0-4e83-b896-9704aeca8926 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1366 microversion: 2.1 time: 0.150908 Sep 18 06:55:40.472876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1233/2301] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:40 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:40.494944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:40.495560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:40.501609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:40.502096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:40.502536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:40.503260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:40.503713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:40.504133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:40.629520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:40.637672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a29b9a43-efa7-4a16-896b-44a28c62fa65 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146786 Sep 18 06:55:40.638240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1234/2302] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:40.750479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:40.751124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:40.757362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:40.757843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:40.758272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:40.758993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:40.759447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:40.759870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:40.908693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:40.916852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0fdbc125-9ed4-4fb0-87da-5af9a9f5cf93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.171356 Sep 18 06:55:40.917424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1235/2303] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:40 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:41.099541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-c9c16e61-e2ee-4818-908f-942f7892efac req-3a5cf14c-5338-4b31-b645-e378bcf4dc58 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ceaad6c7-d3bd-45ed-9e68-010c68b41f1b", "name": "network-vif-plugged", "status": "completed", "tag": "0a70bc98-af92-4192-933a-5024fbdfa5f0"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:55:41.109368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-c9c16e61-e2ee-4818-908f-942f7892efac req-3a5cf14c-5338-4b31-b645-e378bcf4dc58 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:41.109841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-c9c16e61-e2ee-4818-908f-942f7892efac req-3a5cf14c-5338-4b31-b645-e378bcf4dc58 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:41.110274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-c9c16e61-e2ee-4818-908f-942f7892efac req-3a5cf14c-5338-4b31-b645-e378bcf4dc58 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:41.121086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-c9c16e61-e2ee-4818-908f-942f7892efac req-3a5cf14c-5338-4b31-b645-e378bcf4dc58 service nova] Creating event network-vif-plugged:0a70bc98-af92-4192-933a-5024fbdfa5f0 for instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b on hv-cinder-85535 Sep 18 06:55:41.126623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-c9c16e61-e2ee-4818-908f-942f7892efac req-3a5cf14c-5338-4b31-b645-e378bcf4dc58 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.208027 Sep 18 06:55:41.127193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1236/2304] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:55:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:55:41.496066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:41.496860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:41.503245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:41.503730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:41.504413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:41.505108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:41.505565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:41.505991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:41.536342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:41.634115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:41.641974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6b6c893d-cff6-4529-a745-a44c609945e0 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1366 microversion: 2.1 time: 0.150570 Sep 18 06:55:41.642531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1237/2305] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:41 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:41.661025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:41.661623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:41.667235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:41.667720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:41.668178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:41.668827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:41.669275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:41.669697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:41.806146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:41.814268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d0950de8-7ffd-4026-9fed-adcdee735c6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.156752 Sep 18 06:55:41.814845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1238/2306] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:41.939405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:41.940009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:41.946084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:41.946557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:41.947083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:41.947759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:41.948202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:41.948622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:42.074537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:42.083653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d653a8de-b4f4-4510-ada7-a6bea94d44ce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147846 Sep 18 06:55:42.084207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1239/2307] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:41 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:42.665200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:42.665850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:42.667635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=12 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:42.673747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:42.674223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:42.674676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:42.675355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:42.675801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:42.676219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:42.810063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:42.818514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e0f983e4-3444-449b-beec-27ea46fb5ce1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1556 microversion: 2.1 time: 0.157152 Sep 18 06:55:42.819114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1240/2308] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:42 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1556 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:42.836877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:42.837478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:42.843309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:42.843786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:42.844228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:42.844917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:42.845363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:42.845791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:42.971331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:42.980546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e55376c5-6bf9-475b-9ed7-45c6328a7836 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147416 Sep 18 06:55:42.981121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1241/2309] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:42.986108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:42.986600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:42.992086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:42.992602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:42.993033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:42.993709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:42.994154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:42.994571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:43.069615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5caa9f72-af55-4f10-9798-860bb87b3431 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.087413 Sep 18 06:55:43.070204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1242/2310] 10.222.0.20 () {58 vars in 1275 bytes} [Sun Sep 18 06:55:42 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b/os-interface => generated 284 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:55:43.106698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:43.107317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:43.113713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:43.114193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:43.114655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:43.115331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:43.115781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:43.116202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:43.353401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:43.362842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e680021f-3ae1-47ff-81b8-6ecbecf6d5e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.259869 Sep 18 06:55:43.363428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1243/2311] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:43 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:44.004989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:44.005672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:44.012000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:44.012476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:44.012908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:44.013588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:44.014035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:44.014456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:44.164510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:44.173477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eb711d50-58ec-4959-aa46-1b788ffed103 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.173040 Sep 18 06:55:44.174088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1244/2312] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:44.179779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-f07ab3a1-22d4-4074-8590-e921991ddacd req-40ed87fd-8c40-4c6c-b8c6-1e8e4f532728 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ceaad6c7-d3bd-45ed-9e68-010c68b41f1b", "tag": "0a70bc98-af92-4192-933a-5024fbdfa5f0"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:55:44.189445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-f07ab3a1-22d4-4074-8590-e921991ddacd req-40ed87fd-8c40-4c6c-b8c6-1e8e4f532728 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:44.189993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-f07ab3a1-22d4-4074-8590-e921991ddacd req-40ed87fd-8c40-4c6c-b8c6-1e8e4f532728 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:44.190422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-f07ab3a1-22d4-4074-8590-e921991ddacd req-40ed87fd-8c40-4c6c-b8c6-1e8e4f532728 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:44.201275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-f07ab3a1-22d4-4074-8590-e921991ddacd req-40ed87fd-8c40-4c6c-b8c6-1e8e4f532728 service nova] Creating event network-changed:0a70bc98-af92-4192-933a-5024fbdfa5f0 for instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b on hv-cinder-85535 Sep 18 06:55:44.206036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-f07ab3a1-22d4-4074-8590-e921991ddacd req-40ed87fd-8c40-4c6c-b8c6-1e8e4f532728 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030072 Sep 18 06:55:44.206664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1245/2313] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:55:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:55:44.386288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:44.387447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:44.393490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:44.393922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:44.394570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:44.395077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:44.395528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:44.395949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:44.624515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:44.633539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fe6d1b51-534f-4e7b-99e4-07fb8f571b5c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.251540 Sep 18 06:55:44.634308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1246/2314] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:44 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:45.196808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:45.197441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:45.204408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:45.204890 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:45.205326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:45.206012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:45.206465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:45.206916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:45.336321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:45.344550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a154c7f1-fc00-417e-aac6-02c325ae1089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151762 Sep 18 06:55:45.345373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1247/2315] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:45 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:45.656607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:45.657216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:45.663326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:45.663805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:45.664243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:45.664921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:45.665370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:45.665790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:45.669763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:45.789860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:45.798603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc73e4c1-91a8-4698-9f39-c1577813e823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.145913 Sep 18 06:55:45.799173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1248/2316] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:45 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:46.368013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:46.368683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:46.375767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:46.376253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:46.376696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:46.377382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:46.377836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:46.378294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:46.517170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:46.525349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d86b98ed-33b5-4f96-8b27-4724a16613e3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.161220 Sep 18 06:55:46.525918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1249/2317] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:46.823273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:46.823882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:46.829820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:46.830249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:46.830711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:46.831398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:46.831906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:46.832695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:46.963347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:46.972147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d4283e70-789b-426f-96ef-ccdd860f35a5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152717 Sep 18 06:55:46.972717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1250/2318] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:46 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:47.548093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:47.548716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:47.555095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:47.555570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:47.556001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:47.556698 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:47.557147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:47.557570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:47.688927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:47.697165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3a5843c9-ea86-482e-a9b8-7047cf3074b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152952 Sep 18 06:55:47.697725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1251/2319] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:47.997349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:47.998142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:48.002089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:48.006192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:48.006818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:48.007385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:48.008149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:48.008638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:48.009372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:48.134802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:48.142984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5f6d9336-aa22-4a00-b44d-a8b5fffa8011 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149792 Sep 18 06:55:48.143548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1252/2320] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:47 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:55:48.721497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:48.722124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:48.728498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:48.728979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:48.729417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:48.730099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:48.730636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:48.731073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:48.856342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:48.864641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-309480d1-b885-48ab-b0ce-802c11e5b82d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147696 Sep 18 06:55:48.865200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1253/2321] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:49.166519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:49.167150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:49.173067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:49.173544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:49.173981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:49.174687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:49.175135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:49.175555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:49.302845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:49.311073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d55c0aed-7d83-499b-84a3-21f2afb66279 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148433 Sep 18 06:55:49.311643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1254/2322] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:49 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:49.888643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:49.889287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:49.895762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:49.896271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:49.896706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:49.897381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:49.897830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:49.898253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:50.025242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:50.034103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-475b80cf-c41f-4378-bc03-00e4fca5d5aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149425 Sep 18 06:55:50.034673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1255/2323] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:50.335049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:50.335656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:50.341512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:50.341987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:50.342419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:50.343116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:50.344200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:50.344625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:50.475146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:50.483305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8baa1d6d-364e-469b-bcca-dbd72173af52 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152116 Sep 18 06:55:50.483872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1256/2324] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:50 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:51.056138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:51.056832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:51.063940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:51.064431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:51.064871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:51.065558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:51.066013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:51.066444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:51.198934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:51.215570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-74fe1cb2-291c-4ad4-893f-260bb5a39c54 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.163107 Sep 18 06:55:51.216325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1257/2325] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:51.506048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:51.506691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:51.512588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:51.513065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:51.513494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:51.514168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:51.514703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:51.515126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:51.639719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:51.648455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8c828f55-0321-4759-83a6-ef55ed9d04f9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146523 Sep 18 06:55:51.649013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1258/2326] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:51 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:52.242266 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:52.243175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:52.249591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:52.250625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:52.251060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:52.251742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:52.252191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:52.252610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:52.385576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:52.393823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f9314fde-d113-46c0-bace-d3aad8413b36 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.156551 Sep 18 06:55:52.394393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1259/2327] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:52 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:52.672288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:52.672895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:52.679659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:52.680142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:52.680574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:52.681255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:52.681701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:52.682122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:52.808034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:52.816422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-49cd1114-e17c-40dd-a322-3376dc797782 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147970 Sep 18 06:55:52.818207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1260/2328] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:52 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:53.416921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:53.417538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:53.424004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:53.424488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:53.424929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:53.425619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:53.426076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:53.426506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:53.556624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:53.565334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a583d243-5a58-481e-b2d0-1b2525474f28 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152360 Sep 18 06:55:53.565905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1261/2329] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:53.839631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:53.840278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:53.846088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:53.846564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:53.847025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:53.847711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:53.848279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:53.848701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:54.305895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:54.314832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b1eadc1b-024f-4b80-b9bc-6dc5833bfe7f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.479117 Sep 18 06:55:54.315404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1262/2330] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:53 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:54.588576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:54.589253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:54.597008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:54.597499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:54.597944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:54.598675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:54.599134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:54.599567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:54.726228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:54.734893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7fa52d46-e57c-4fb1-9307-23b7ee3eff0b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150265 Sep 18 06:55:54.735454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1263/2331] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:55.339086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:55.339632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:55.346047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:55.346525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:55.346986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:55.347665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:55.348110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:55.348530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:55.475327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:55.478759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:55.484388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-852af626-6a11-4bb5-837a-19b3eef2db11 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149290 Sep 18 06:55:55.484950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1264/2332] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:55 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:55.758682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:55.759340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:55.766337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:55.766841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:55.767319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:55.767989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:55.768440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:55.768857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:55.900729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:55.910269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3927872a-5132-4116-b496-428d495df59d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.155411 Sep 18 06:55:55.910887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1265/2333] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:56.508761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:56.509751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:56.515813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:56.516519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:56.517409 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:56.518277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:56.518768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:56.519190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:56.649263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:56.657937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f6668e10-de1f-414a-8e3d-b1e8d5f9e1f0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154589 Sep 18 06:55:56.658498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1266/2334] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:56 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:56.933297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:56.933992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:56.940283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:56.940759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:56.941203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:56.941880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:56.942332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:56.942783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:57.072064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:57.081105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ee9edbbe-256d-4055-a3cf-3341cdf63994 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151741 Sep 18 06:55:57.081739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1267/2335] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:56 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:57.681234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:57.681848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:57.688419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:57.688899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:57.689335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:57.690017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:57.690467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:57.690919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:57.815949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:57.824061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-07c45550-c7f1-4b01-b227-2dffbf3c3287 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146712 Sep 18 06:55:57.824630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1268/2336] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:57 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:58.104237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:58.104872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:58.111696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:58.112177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:58.112609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:58.113283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:58.113732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:58.114155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:58.239105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:58.247222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-21ba0762-8859-464d-a7ed-eab776b538f5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146894 Sep 18 06:55:58.247779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1269/2337] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:58 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:58.846650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:58.847181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:58.853124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:58.853598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:58.854030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:58.854734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:58.855180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:58.855598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:58.984498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:58.992604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a3b52092-9819-46c8-b551-9944f3e70fc2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149914 Sep 18 06:55:58.993173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1270/2338] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:58 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:55:59.269732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:55:59.270400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:55:59.276690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:59.277168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:59.277603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:59.278287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:55:59.278770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:55:59.279193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:55:59.407674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:55:59.415936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b0dec417-fd3f-46e0-a7b7-e45db4a1d0f0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149985 Sep 18 06:55:59.416502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1271/2339] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:55:59 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:00.016264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:00.016877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:00.022895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:00.023383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:00.023820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:00.024506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:00.024963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:00.025387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:00.157274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:00.165618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-92451e2f-efaf-4d1c-a6de-fa7a342b23c5 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.153751 Sep 18 06:56:00.166220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1272/2340] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:00 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:00.443539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:00.444200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:00.450588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:00.451126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:00.451564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:00.452251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:00.453359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:00.453791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:00.582336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:00.590796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-822d678f-da99-48d8-8325-fe9f945b345d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151127 Sep 18 06:56:00.591365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1273/2341] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:01.188702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:01.189312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:01.195307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:01.195783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:01.196216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:01.196893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:01.197368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:01.197789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:01.329351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:01.337483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-332d933d-c67f-402c-b255-3267c50184a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152810 Sep 18 06:56:01.338045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1274/2342] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:01 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:01.613870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:01.614597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:01.620904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:01.621380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:01.621808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:01.622485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:01.622962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:01.623381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:01.751902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:01.760365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-37eee53e-f19d-45f7-a2f3-241a9870be27 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150371 Sep 18 06:56:01.760929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1275/2343] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:02.361434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:02.362046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:02.368075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:02.368558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:02.368996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:02.369681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:02.370135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:02.370560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:02.497733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:02.506245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8871cf0e-41bd-43ce-9460-368c26620c92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149242 Sep 18 06:56:02.506843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1276/2344] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:02 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:02.784529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:02.785186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:02.791504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:02.791987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:02.792428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:02.793113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:02.793566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:02.793993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:02.922548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:02.930836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-22babc6a-1b79-4c82-8682-14687f242fcd tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150656 Sep 18 06:56:02.931403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1277/2345] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:03.529289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:03.529908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:03.535883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:03.536366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:03.536806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:03.537514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:03.538006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:03.538436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:03.667466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:03.676102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9371e732-0413-4551-aaf7-6b2b3265505a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150666 Sep 18 06:56:03.676670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1278/2346] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:03 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:03.954243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:03.954916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:03.961223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:03.961699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:03.962131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:03.962901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:03.963349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:03.963766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:04.093846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:04.102010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-35e44e79-e614-4b36-bec6-cc3865b6e939 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151431 Sep 18 06:56:04.102571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1279/2347] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:04.699169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:04.699823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:04.706376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:04.706860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:04.707292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:04.707964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:04.708408 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:04.708831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:04.833155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:04.841205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b81f0336-2b95-4e19-887e-7946726426f4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.145885 Sep 18 06:56:04.841774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1280/2348] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:04 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:05.125966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:05.126622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:05.133223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:05.133702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:05.134134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:05.134846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:05.135317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:05.135736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:05.153135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:56:05.263212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:05.271422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-28d93dd7-af16-46af-a5ca-65d3c264772b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150085 Sep 18 06:56:05.271981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1281/2349] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:05.304682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:56:05.305175 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:56:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:56:05.863411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:05.864027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:05.870246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:05.870773 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:05.871220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:05.871912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:05.872367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:05.872796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:06.005285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:06.013656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-961de303-053a-4dfa-975e-f7f8eb4f8437 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154132 Sep 18 06:56:06.014228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1282/2350] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:05 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:06.294713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:06.295324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:06.301724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:06.302206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:06.302677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:06.303362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:06.303816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:06.304238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:06.438074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:06.446206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-31434eee-6c94-4581-8430-15531426a085 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.155357 Sep 18 06:56:06.446794 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1283/2351] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:07.037325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:07.037934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:07.043908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:07.044392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:07.044826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:07.045506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:07.046570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:07.047019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:07.174714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:07.182926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e96c1a57-8afd-4810-bf73-b465ff58d39a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149474 Sep 18 06:56:07.183498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1284/2352] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:07 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:07.469636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:07.470254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:07.476682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:07.477158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:07.477594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:07.478273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:07.478764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:07.479191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:07.606830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:07.615774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c6fb2e44-cafb-4162-844f-963257579ffa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149907 Sep 18 06:56:07.616284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1285/2353] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:08.206333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:08.206971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:08.212975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:08.213456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:08.213891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:08.214592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:08.215050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:08.215477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:08.349696 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:08.356782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e06410eb-d446-4f98-916b-e2a2349e7f51 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154260 Sep 18 06:56:08.357351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1286/2354] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:08 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:08.639757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:08.640407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:08.646953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:08.647435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:08.647876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:08.648559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:08.649013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:08.649439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:08.778934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:08.787635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dbd6d9a2-f4f8-4d68-b025-e57ba4ea5063 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151776 Sep 18 06:56:08.788207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1287/2355] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:09.381645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:09.382261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:09.388370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:09.388851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:09.389289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:09.390713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:09.391174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:09.391628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:09.517722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:09.525907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ed937218-2297-429c-bd52-2d898d301c2d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148824 Sep 18 06:56:09.526481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1288/2356] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:09 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:09.810647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:09.811295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:09.818190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:09.818693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:09.819125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:09.819801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:09.820248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:09.820667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:09.950013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:09.958129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ed1ae6eb-8d54-4f44-a03b-d8fc33a0e198 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151348 Sep 18 06:56:09.958724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1289/2357] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:10.551371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:10.551984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:10.558131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:10.558636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:10.559073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:10.559743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:10.560189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:10.560607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:10.694128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:10.702254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a39da51d-09ed-43f6-9176-dff4142a35c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.156985 Sep 18 06:56:10.702879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1290/2358] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:10 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:10.982145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:10.982831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:10.989728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:10.990210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:10.990684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:10.991370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:10.991822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:10.992245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:11.129203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:11.137640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-32dabeac-cd4e-4c99-8bc0-a27646122133 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.159288 Sep 18 06:56:11.138216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1291/2359] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:11.725647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:11.726265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:11.732276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:11.732752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:11.733189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:11.734362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:11.734841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:11.735265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:11.863901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:11.878176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e226619c-b5bc-4210-bb23-37861f011f69 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.156358 Sep 18 06:56:11.879148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1292/2360] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:11 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:12.161701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:12.162378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:12.168693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:12.169166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:12.169597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:12.170369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:12.170855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:12.171302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:12.299354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:12.308185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-40f1cc00-f386-4857-b638-424597cf5395 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150327 Sep 18 06:56:12.308749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1293/2361] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:12.901407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:12.902053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:12.908144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:12.908632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:12.909070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:12.909755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:12.910209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:12.910666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:13.038234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:13.046381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c151a56b-3de2-4d8d-ab46-2897a5e35f75 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148863 Sep 18 06:56:13.046982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1294/2362] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:12 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:13.330384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:13.331050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:13.337355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:13.337841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:13.338280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:13.339004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:13.339464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:13.339928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:13.467711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:13.475958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-123e9028-c3ca-4810-82c6-b99e8a79f3e4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149349 Sep 18 06:56:13.476523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1295/2363] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:13 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:14.069597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:14.070207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:14.076185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:14.076665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:14.077100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:14.078314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:14.078823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:14.079251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:14.210205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:14.218474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-828452d0-b939-4228-98f3-3ae8b65e0fba tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152658 Sep 18 06:56:14.219071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1296/2364] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:14 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:14.499301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:14.499933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:14.506528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:14.507041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:14.507475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:14.508158 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:14.508610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:14.509038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:14.635285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:14.643519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cc0965c1-3d23-45ca-8271-9eb616058479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148122 Sep 18 06:56:14.644086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1297/2365] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:15.241887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:15.242494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:15.248424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:15.248898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:15.249333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:15.250004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:15.250453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:15.250898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:15.255054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:56:15.379655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:15.388070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-503b894b-f5b7-462a-a5a3-5987cef2b823 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149997 Sep 18 06:56:15.388629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1298/2366] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:15 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:15.666942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:15.667568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:15.673929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:15.674407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:15.674943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:15.675620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:15.676067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:15.676489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:15.802149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:15.810412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-61c89d14-a560-482f-9b64-c64ae9fe5687 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147261 Sep 18 06:56:15.811009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1299/2367] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:16.411010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:16.411547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:16.417652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:16.418133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:16.418570 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:16.419269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:16.419722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:16.420173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:16.546988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:16.555378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1ae832d8-9b41-4d10-926e-0a07e93ac689 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148252 Sep 18 06:56:16.555962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1300/2368] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:16 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:16.833697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:16.834805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:16.841139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:16.841628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:16.842063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:16.842796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:16.843800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:16.844257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:16.985819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:16.994800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0bee887c-83da-4dca-bf4e-8b3fc9a267cf tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.165361 Sep 18 06:56:16.995375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1301/2369] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:17.579521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:17.580132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:17.583120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:56:17.587151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:17.587638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:17.588080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:17.588762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:17.589213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:17.589638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:17.716069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:17.724336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-85bef599-01d6-427b-a080-cff908ac263a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149394 Sep 18 06:56:17.724908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1302/2370] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:17 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:18.018684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:18.019307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:18.025564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:18.026048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:18.026493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:18.027204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:18.027670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:18.028098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:18.161343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:18.171616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3fd82921-df35-40d3-bf0f-61149307dac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.156091 Sep 18 06:56:18.172211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1303/2371] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:18.747912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:18.748431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:18.754996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:18.755484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:18.755920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:18.756605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:18.757083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:18.757505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:18.883466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:18.892229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7ecc5e56-8138-4aa9-8a4f-f4f286af18fd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148328 Sep 18 06:56:18.892793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1304/2372] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:18 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:18.897125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:18.897609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:18.903026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:18.903497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:18.903927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:18.904598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:18.905042 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:18.905460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:18.927762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-37f4d200-895f-4b18-a170-7a51b67e3c45 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b/ips" status: 200 len: 149 microversion: 2.1 time: 0.034014 Sep 18 06:56:18.928333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1305/2373] 10.222.0.20 () {58 vars in 1248 bytes} [Sun Sep 18 06:56:18 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b/ips => generated 149 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:56:19.194615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:19.195279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:19.201047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:19.201514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:19.201948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:19.203233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:19.203683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:19.204103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:19.327438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:19.335577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f59a39b4-138f-4fa6-8cb6-6ffc22587b6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.144670 Sep 18 06:56:19.336146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1306/2374] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:19.915316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:19.915922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:19.923150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:19.923634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:19.924070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:19.924756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:19.925239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:19.925663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:20.052964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:20.061073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-70c2be45-1694-4fb5-9ece-d14a74822992 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149609 Sep 18 06:56:20.061671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1307/2375] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:19 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:20.358292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:20.359032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:20.364866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:20.365345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:20.365776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:20.366452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:20.366935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:20.367355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:20.514085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:20.522800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2a80187f-7e9c-49e2-b989-1caa1fbd1f84 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.168715 Sep 18 06:56:20.523366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1308/2376] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:20 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:21.084032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:21.084638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:21.091087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:21.091562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:21.091993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:21.092667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:21.093118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:21.093533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:21.221861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:21.230002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0cd0c2ee-c480-4f92-8d81-ce36394170d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149796 Sep 18 06:56:21.230564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1309/2377] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:21 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:21.545088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:21.545720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:21.551648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:21.552122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:21.552553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:21.553222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:21.553666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:21.554083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:21.679114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:21.687194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c86f2673-aa8b-4101-9454-f6f58a296743 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.145939 Sep 18 06:56:21.687761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1310/2378] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:22.255416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:22.255950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:22.262538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:22.263048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:22.263492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:22.264189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:22.264684 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:22.265111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:22.393410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:22.401544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6a53728a-9453-4a44-880a-3befc6de01b0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150389 Sep 18 06:56:22.402119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1311/2379] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:22 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:22.710647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:22.711260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:22.717203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:22.718178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:22.718642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:22.719325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:22.719787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:22.720215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:22.850395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:22.859199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-084b3c7f-f70f-4147-968b-b72766e5e4de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151806 Sep 18 06:56:22.859765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1312/2380] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:22 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:23.425560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:23.426217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:23.432759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:23.433246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:23.433687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:23.434372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:23.434853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:23.435283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:23.563054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:23.571296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3abf005c-9037-42db-ac9f-dc34fc4cec93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150249 Sep 18 06:56:23.571854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1313/2381] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:23 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:23.882329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:23.883066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:23.888946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:23.889949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:23.890388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:23.891113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:23.891568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:23.891995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:24.026671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:24.035520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c4a58597-b378-4ad2-9874-65653a4b786f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.157271 Sep 18 06:56:24.036100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1314/2382] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:24.594358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:24.594996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:24.601470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:24.601955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:24.602390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:24.603100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:24.603550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:24.603974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:24.607840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:56:24.730677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:24.739290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-22b09a19-baef-4db9-b462-d477dca1c691 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149000 Sep 18 06:56:24.739862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1315/2383] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:24 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:25.057978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:25.058645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:25.064437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:25.064922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:25.065358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:25.066040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:25.066493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:25.066946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:25.195631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:25.203801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dd76bf44-31b6-465d-b5a0-1cd396963ec6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149677 Sep 18 06:56:25.204369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1316/2384] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:25.763010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:25.763648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:25.770956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:25.771434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:25.771893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:25.772572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:25.773018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:25.773436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:25.899942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:25.908252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8b780e14-f528-4afd-822b-9439ca09e599 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148988 Sep 18 06:56:25.908810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1317/2385] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:25 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:26.227322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:26.228010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:26.234549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:26.234972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:26.235404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:26.236078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:26.236535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:26.236952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:26.360526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:26.368655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2ec2b71c-276d-4828-afac-96a53e19d3c2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.145200 Sep 18 06:56:26.369218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1318/2386] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:26.435147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:56:26.435697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:56:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:56:26.436712 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:56:26.436892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:56:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:56:26.931094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:26.931701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:26.938380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:26.938919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:26.939372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:26.940067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:26.940526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:26.940956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:27.074424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:27.083309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7691ae5c-fa9f-4123-8975-bf75aeb71c24 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.156041 Sep 18 06:56:27.083881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1319/2387] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:26 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:27.395418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:27.398161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:27.406455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:27.406965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:27.407395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:27.408076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:27.408523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:27.408946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:27.535351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:27.543440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1dc5a88c-24ac-49b1-893f-85d041d67b29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154830 Sep 18 06:56:27.544004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1320/2388] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:28.107857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:28.108955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:28.115382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:28.115858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:28.116287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:28.116967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:28.117411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:28.117830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:28.248315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:28.256658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fa1ba4b5-589f-470b-a7b5-02c152f7a242 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152766 Sep 18 06:56:28.257219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1321/2389] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:28 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:28.566753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:28.567358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:28.573288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:28.573762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:28.574190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:28.574897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:28.575344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:28.575764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:28.700392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:28.709182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-feb98e19-fb89-4312-872f-f4f90a36ea23 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146272 Sep 18 06:56:28.709755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1322/2390] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:28 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:29.280034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:29.280677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:29.287275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:29.287756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:29.288191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:29.288878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:29.289331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:29.289765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:29.418714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:29.427447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5ba6a1af-6894-4ef2-bcc1-52184672b9c3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151387 Sep 18 06:56:29.428019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1323/2391] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:29 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:29.732395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:29.733595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:29.739539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:29.740011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:29.740437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:29.741112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:29.741557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:29.741977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:29.867448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:29.875549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-381c7c1d-481a-488a-bcb1-6bdd0bf2afc7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147007 Sep 18 06:56:29.876111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1324/2392] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:30.450086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:30.450759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:30.458079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:30.458556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:30.459019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:30.459704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:30.460153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:30.460573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:30.586957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:30.595356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f4342de3-2053-4aa9-8ac5-39d785079fda tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149066 Sep 18 06:56:30.595927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1325/2393] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:30 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:30.899388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:30.900013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:30.906411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:30.906932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:30.907376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:30.908066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:30.908528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:30.908958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:31.037019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:31.045224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-411ef3e6-0888-4f85-be21-263ddcb314b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149657 Sep 18 06:56:31.046381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1326/2394] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:30 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:31.618557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:31.619190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:31.626138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:31.626557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:31.627021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:31.627693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:31.628141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:31.628827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:31.756527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:31.765316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8e609115-a57a-4c2d-a301-89dc88d95318 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150596 Sep 18 06:56:31.765877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1327/2395] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:31 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:32.071174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:32.071835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:32.077588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:32.078059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:32.078492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:32.079197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:32.079646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:32.080063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:32.208146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:32.216436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6751cec8-3450-44ea-9a3f-aaf7c24b6aa7 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149034 Sep 18 06:56:32.217015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1328/2396] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:32 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:32.788097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:32.788626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:32.795078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:32.795578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:32.796009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:32.796679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:32.797122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:32.797538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:32.924388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:32.932559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-080f6398-04ea-4c12-988d-61eb7742ec34 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148384 Sep 18 06:56:32.933118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1329/2397] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:32 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:33.239414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:33.240057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:33.245778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:33.246254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:33.246712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:33.247388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:33.247832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:33.248249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:33.374511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:33.382901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-81ef838b-7855-4d65-b701-7f316c76a089 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147154 Sep 18 06:56:33.383458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1330/2398] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:33 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:33.956353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:33.956954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:33.963462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:33.963935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:33.964360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:33.965033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:33.965478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:33.965898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:34.095669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:34.099065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:56:34.104548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8db6c614-b706-44eb-9bde-c24c91d89538 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152072 Sep 18 06:56:34.105115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1331/2399] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:33 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:34.406631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:34.407275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:34.413268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:34.413742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:34.414173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:34.414872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:34.415318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:34.415739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:34.543032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:34.551223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f46bb7f6-0933-4f2a-9c8a-ac1bc435e9b5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148989 Sep 18 06:56:34.551781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1332/2400] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:35.128709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:35.129327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:35.135838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:35.136324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:35.136766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:35.137450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:35.137906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:35.138332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:35.273534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:35.281889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0d16825a-565b-4575-aff0-1aa7fbef6286 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.157358 Sep 18 06:56:35.282463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1333/2401] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:35 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:35.574236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:35.574946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:35.580849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:35.581323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:35.581769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:35.582455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:35.582945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:35.583367 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:35.713782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:35.722699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8e63a654-9636-4954-9b4e-6b442c3ce3fe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152228 Sep 18 06:56:35.723207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1334/2402] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:36.305450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:36.306057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:36.313091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:36.313569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:36.314000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:36.314717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:36.315169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:36.315602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:36.445983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:36.454862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f9261fea-a43e-4d05-a599-3b4b535282ed tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.153196 Sep 18 06:56:36.455501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1335/2403] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:36 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:36.746140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:36.746821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:36.752643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:36.753119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:36.753809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:36.754489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:36.754958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:36.755426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:36.880499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:36.888739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1ff2f77c-e902-4c5e-bfbb-ed49fbf4f0cb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146399 Sep 18 06:56:36.889883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1336/2404] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:37.478488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:37.479137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:37.485995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:37.486469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:37.486925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:37.487598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:37.488043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:37.488458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:37.617751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:37.626753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-31df94a4-f7c5-4e79-803f-181196a325ae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152049 Sep 18 06:56:37.627256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1337/2405] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:37 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:37.912585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:37.913263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:37.919103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:37.919591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:37.920633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:37.921312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:37.921766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:37.922191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:38.052070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:38.060273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f3fa0488-df76-4a4c-b82e-0b9f985a79d6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151490 Sep 18 06:56:38.060843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1338/2406] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:37 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:38.649909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:38.650523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:38.656993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:38.657470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:38.657903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:38.658599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:38.659055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:38.659478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:38.787760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:38.796653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c4303062-c87e-4d19-96da-8853513f6e4c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150565 Sep 18 06:56:38.797227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1339/2407] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:38 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:39.084507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:39.085181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:39.091026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:39.091514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:39.092596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:39.093319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:39.093780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:39.094210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:39.223366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:39.231534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b7a2a767-1eb8-4d65-af57-c8452936bc5c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150991 Sep 18 06:56:39.232108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1340/2408] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:39 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:39.820599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:39.821211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:39.827626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:39.828112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:39.828548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:39.829228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:39.829677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:39.830102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:39.956859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:39.964942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aea13b77-80fb-463e-b876-21a4bdf0144a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148203 Sep 18 06:56:39.965506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1341/2409] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:39 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:40.254928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:40.255591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:40.261486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:40.261963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:40.262397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:40.263100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:40.264034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:40.264462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:40.389728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:40.398012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d8bc2643-f13b-49d6-9603-335bcaa05166 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147064 Sep 18 06:56:40.398786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1342/2410] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:40.988641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:40.989247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:40.995709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:40.996183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:40.996614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:40.997287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:40.997735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:40.998799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:41.129502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:41.137630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-baaa1747-4470-499e-8cbb-909426789bf4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152888 Sep 18 06:56:41.138198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1343/2411] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:40 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:41.422023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:41.422656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:41.428610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:41.429093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:41.429533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:41.430226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:41.430708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:41.431259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:41.560903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:41.569173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b7370d17-5326-430b-a42b-a155477b67d9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151556 Sep 18 06:56:41.569737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1344/2412] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:42.160848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:42.161457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:42.167914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:42.168391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:42.168820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:42.169501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:42.169952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:42.170376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:42.298736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:42.306908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-090fe204-ac07-4f26-9a38-063f182c326c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149971 Sep 18 06:56:42.307478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1345/2413] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:42 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:42.592488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:42.593118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:42.599043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:42.599526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:42.599961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:42.600643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:42.601096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:42.601522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:42.729185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:42.738898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-41d890c0-7491-49bb-8caf-3936de0ec711 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150158 Sep 18 06:56:42.739282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1346/2414] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:43.331109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:43.331717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:43.338167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:43.338668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:43.339103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:43.339776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:43.340248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:43.340663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:43.475988 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:43.484151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-df03654c-8320-40ac-8c76-a5a390a1ab1f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.156912 Sep 18 06:56:43.484708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1347/2415] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:43 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:43.762635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:43.763290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:43.769711 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:43.770193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:43.770728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:43.771418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:43.771871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:43.772298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:43.789659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:56:43.904629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:43.912757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-01c3e924-67d3-4aa7-8637-50d9bf420f25 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.153940 Sep 18 06:56:43.913324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1348/2416] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:44.508464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:44.509104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:44.515604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:44.516084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:44.516516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:44.517190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:44.517636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:44.518055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:44.642422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:44.650742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e22a160a-a430-4bbc-b40a-3d223ebb051b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146049 Sep 18 06:56:44.651311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1349/2417] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:44 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:44.936177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:44.936826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:44.942742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:44.943225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:44.943658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:44.944343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:44.944796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:44.945218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:45.075285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:45.084149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7e6032ff-7c6f-4bf5-b263-7f82c143d734 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151909 Sep 18 06:56:45.084724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1350/2418] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:45.674818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:45.675452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:45.681813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:45.682288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:45.682748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:45.683421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:45.683867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:45.684286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:45.808828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:45.817625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cf7ac554-8f19-4006-87e5-51c620f52dce tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146623 Sep 18 06:56:45.818182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1351/2419] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:45 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:46.108427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:46.109052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:46.112039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:56:46.116222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:46.116706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:46.117143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:46.117832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:46.118285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:46.118752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:46.246114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:46.254655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0f97deaf-0d8d-4626-8320-28b8e4d80624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150797 Sep 18 06:56:46.255226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1352/2420] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:46.841246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:46.841938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:46.849269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:46.849749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:46.850183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:46.850899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:46.851347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:46.851770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:46.978833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:46.987462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fbe4de1e-b13f-4707-9a8a-60a5e3f4f761 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150920 Sep 18 06:56:46.988030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1353/2421] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:46 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:47.277660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:47.278315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:47.284430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:47.284910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:47.285342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:47.286044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:47.286491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:47.286982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:47.412127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:47.421209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a21c0f0d-c772-46a6-8088-4609c6cd2bbb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147361 Sep 18 06:56:47.421767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1354/2422] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:48.014088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:48.014478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:48.020943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:48.021425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:48.021864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:48.022553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:48.023031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:48.023455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:48.155145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:48.163369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-84ec7cc7-b502-48af-b7aa-c1526d38436d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.155642 Sep 18 06:56:48.163944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1355/2423] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:48 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:48.444802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:48.445472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:48.451358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:48.452483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:48.452920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:48.453605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:48.454058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:48.454480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:48.580436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:48.588652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e97c821c-2fc5-4c94-b894-07299522974b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147710 Sep 18 06:56:48.589218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1356/2424] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:49.187195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:49.187801 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:49.193682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:49.194156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:49.194594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:49.195276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:49.195717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:49.196133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:49.325318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:49.333417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fd0f1bcc-53e4-4fb9-ad3f-021cfade7f00 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150709 Sep 18 06:56:49.333976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1357/2425] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:49 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:49.612276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:49.612907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:49.618850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:49.619329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:49.619768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:49.620446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:49.620894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:49.621314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:49.746517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:49.754994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e11aa60f-aceb-49fe-a747-8ef161a1740b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147181 Sep 18 06:56:49.755558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1358/2426] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:50.356462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:50.357096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:50.364853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:50.365330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:50.365792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:50.366469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:50.366948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:50.367371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:50.498276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:50.506750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-52228e52-5be8-43f0-9ee2-6408c7bfecca tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154104 Sep 18 06:56:50.507311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1359/2427] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:50 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:50.777901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:50.778537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:50.785146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:50.785631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:50.786068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:50.786789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:50.787241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:50.787665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:50.913148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:50.921415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-45cb8790-b0fc-433e-850c-045e0efd4cdc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147381 Sep 18 06:56:50.921981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1360/2428] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:51.529642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:51.530253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:51.536806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:51.537289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:51.537727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:51.538411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:51.538897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:51.539321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:51.663625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:51.672219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-47fe63ce-900f-4b83-91d1-62cb6a7f8e68 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146485 Sep 18 06:56:51.672782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1361/2429] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:51 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:51.944506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:51.945145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:51.950908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:51.951385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:51.951817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:51.952488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:51.952939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:51.953357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:52.081253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:52.089372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7f3e34d9-fc26-4130-8b9c-76b009521278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148826 Sep 18 06:56:52.089936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1362/2430] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:52.695533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:52.696139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:52.703073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:52.703548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:52.704024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:52.704697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:52.705144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:52.705561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:52.830659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:52.838853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-70f76348-8b3d-4505-87ad-05682c45d122 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147107 Sep 18 06:56:52.839421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1363/2431] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:52 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:53.113277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:53.113904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:53.119703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:53.120190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:53.120630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:53.121309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:53.121757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:53.122179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:53.256049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:53.264518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-14f77d72-a5d1-4782-9a4f-d192b122ce33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.155135 Sep 18 06:56:53.265088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1364/2432] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:53.863156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:53.863882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:53.870406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:53.870984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:53.871419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:53.872104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:53.872555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:53.872977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:53.877044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:56:54.089881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:54.099159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2858eda3-b05d-49ae-a2d5-cf6a9c327815 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.239905 Sep 18 06:56:54.099744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1365/2433] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:53 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:54.287900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:54.288571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:54.294712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:54.295207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:54.296232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:54.296937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:54.297395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:54.298853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:54.631135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:54.639329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ad8eff25-923d-4996-b6f4-daf7f72399c3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.355375 Sep 18 06:56:54.639894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1366/2434] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:55.124799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:55.125413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:55.132132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:55.132613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:55.133051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:55.133735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:55.134191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:55.134642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:55.263609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:55.272285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d7d29170-3beb-497f-876a-4731520a7d5f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151377 Sep 18 06:56:55.272870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1367/2435] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:55 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:55.662213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:55.662885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:55.668752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:55.669228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:55.669660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:55.670332 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:55.670822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:55.671240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:55.799272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:55.807501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ce090387-91ff-4d0f-a84a-978ed6b174df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149102 Sep 18 06:56:55.808061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1368/2436] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:56.301918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:56.302447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:56.308827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:56.309506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:56.310025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:56.311319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:56.312002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:56.312508 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:56.439036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:56.447286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f2e895c4-8730-499a-8e6e-66e71c364521 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149311 Sep 18 06:56:56.447857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1369/2437] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:56 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:56.830947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:56.831636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:56.837499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:56.837983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:56.838420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:56.839149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:56.839602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:56.840026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:56.964903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:56.973733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2b11ce8f-a59f-4f53-858b-ca7f8d242e2d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147190 Sep 18 06:56:56.974301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1370/2438] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:56 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:57.469873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:57.470486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:57.478129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:57.478635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:57.479067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:57.479748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:57.480193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:57.480612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:57.622190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:57.631361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-919dbee4-aba8-4e7c-a745-3fafb880d077 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.165316 Sep 18 06:56:57.631923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1371/2439] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:57 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:57.997140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:57.997770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:58.004201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:58.004692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:58.005127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:58.005816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:58.006297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:58.006751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:58.141251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:58.149528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c09b0470-10e3-4f1c-bc00-6151223d1c83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.156221 Sep 18 06:56:58.150131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1372/2440] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:56:58.653996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:58.654670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:58.661025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:58.661496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:58.661928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:58.662622 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:58.663071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:58.663488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:58.793023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:58.801260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-431d61b7-3580-43d5-ad20-19f1b8f6ba96 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151133 Sep 18 06:56:58.801826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1373/2441] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:58 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:59.173514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:59.174163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:59.180009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:59.180489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:59.180926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:59.181611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:59.182060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:59.182482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:59.306782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:59.315505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c213d80a-a719-455c-8462-407b43625ee3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146310 Sep 18 06:56:59.316069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1374/2442] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:59 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:56:59.824860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:56:59.825405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:56:59.832403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:59.832887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:59.833329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:59.834024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:56:59.834480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:56:59.835004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:56:59.961850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:56:59.970750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3f287fec-3cd9-49a3-b6d8-1eec9388b63e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149812 Sep 18 06:56:59.971320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1375/2443] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:56:59 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:00.337743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:00.338510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:00.344206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:00.344682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:00.345113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:00.345784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:00.346232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:00.346679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:00.480958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:00.489591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0d4d1231-2d68-44f8-915a-7d373c689b92 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.155664 Sep 18 06:57:00.490135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1376/2444] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:00.993633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:00.994237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:01.000626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:01.001124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:01.001560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:01.002232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:01.002703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:01.003381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:01.128353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:01.136420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6c0c24a0-3ec1-4a5f-a686-d392243a0472 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146613 Sep 18 06:57:01.136984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1377/2445] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:00 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:01.514807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:01.515432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:01.521458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:01.521957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:01.522389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:01.523103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:01.523554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:01.523977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:01.651448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:01.659749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-abf31ba3-7fef-4f95-8dc8-7ba654f4c9c8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149714 Sep 18 06:57:01.660314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1378/2446] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:02.159393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:02.160001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:02.166416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:02.166918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:02.167349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:02.168019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:02.168463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:02.168878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:02.294294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:02.302908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ff36cc9b-9d1b-435b-9a74-f329fd1b31d7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147328 Sep 18 06:57:02.303482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1379/2447] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:02 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:02.682738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:02.683431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:02.689354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:02.689824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:02.690253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:02.690954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:02.691402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:02.691823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:02.820077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:02.828641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8c319986-7dce-4af8-9ca9-5ccdd97cf295 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149787 Sep 18 06:57:02.829207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1380/2448] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:03.326867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:03.327469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:03.333843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:03.334316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:03.334788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:03.335463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:03.335909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:03.336329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:03.476436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:03.479885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:03.486322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3127650f-9667-4d0e-b20b-407572e1452a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.163893 Sep 18 06:57:03.486923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1381/2449] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:03 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:03.852052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:03.852739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:03.859456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:03.859945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:03.860374 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:03.861050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:03.861502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:03.861920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:03.986377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:03.994546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6fd10639-8e53-4572-9a71-7154b99b70eb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146302 Sep 18 06:57:03.995130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1382/2450] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:04.509513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:04.510118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:04.517026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:04.517502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:04.517933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:04.518637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:04.519087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:04.519502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:04.651611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:04.659655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-02d870c2-caab-4c25-b913-ed8c0d83a167 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154067 Sep 18 06:57:04.660215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1383/2451] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:04 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:05.017123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:05.017785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:05.023600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:05.024087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:05.024504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:05.025174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:05.025615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:05.026033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:05.150847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:05.158917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0c937955-7f4f-4b0e-9ad2-18fc1cefd0d1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.145553 Sep 18 06:57:05.159484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1384/2452] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:05.682959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:05.683566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:05.689913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:05.690385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:05.690846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:05.691528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:05.691971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:05.692405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:05.821566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:05.830338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-15bf81dc-11c8-4dad-8bcc-e4d070d2ea84 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151177 Sep 18 06:57:05.830933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1385/2453] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:05 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:06.182400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:06.183051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:06.189501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:06.189977 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:06.190412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:06.191138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:06.191600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:06.192021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:06.317009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:06.325195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6a5ecd22-c91a-4c79-bc4b-ab98deef30ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146657 Sep 18 06:57:06.325761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1386/2454] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:06 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:06.853135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:06.853739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:06.860813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:06.861288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:06.861752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:06.862431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:06.862902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:06.863324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:06.997217 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:07.005590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2553c9c5-d139-487a-a16d-a51ac0e519d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.156258 Sep 18 06:57:07.006150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1387/2455] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:06 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:07.349047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:07.349679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:07.355722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:07.356202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:07.356634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:07.357310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:07.357756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:07.358178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:07.488733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:07.496947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-613ca45b-c6df-45d5-8ff8-3d384b87c00f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151764 Sep 18 06:57:07.497503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1388/2456] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:08.027500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:08.028113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:08.034519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:08.035033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:08.035469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:08.036149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:08.036600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:08.037021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:08.162642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:08.171252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7af31bdf-101a-4f9b-a70e-3511302617cd tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147592 Sep 18 06:57:08.171818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1389/2457] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:08 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:08.520065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:08.520679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:08.526706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:08.527188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:08.527624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:08.528304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:08.529241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:08.529667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:08.657908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:08.666036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-60ac0901-ce7e-4020-a120-1a21e836e4ad tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149813 Sep 18 06:57:08.666804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1390/2458] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:09.195587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:09.196193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:09.204864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:09.205342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:09.205776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:09.206460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:09.207236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:09.207953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:09.339649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:09.347727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-368a9ad6-098d-4d69-a7f6-b7947be2e928 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.156749 Sep 18 06:57:09.348288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1391/2459] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:09 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:09.690136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:09.690837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:09.696643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:09.697122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:09.697552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:09.698225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:09.698699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:09.699123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:09.832513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:09.840712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0f03d8a-f481-4051-b7f6-d4a6d4aee771 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154459 Sep 18 06:57:09.841787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1392/2460] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:10.371262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:10.371876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:10.378608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:10.379111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:10.379549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:10.380238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:10.380691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:10.381119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:10.635663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:10.644481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-214d4724-e49b-42a7-9aea-9991abc54f47 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.277095 Sep 18 06:57:10.645040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1393/2461] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:10 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:10.863859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:10.864506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:10.870253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:10.870777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:10.871219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:10.871902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:10.872885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:10.873311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:10.999697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:11.008139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-843aae5d-3753-4f6e-8abf-01ed3ace09ca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148004 Sep 18 06:57:11.008708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1394/2462] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:11.667349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:11.667957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:11.675018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:11.675493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:11.676631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:11.677233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:11.677681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:11.678099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:11.807086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:11.815527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-662748ec-a58b-4ff7-b4ed-d16fe53fa016 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151685 Sep 18 06:57:11.816121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1395/2463] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:11 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:12.031931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:12.032584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:12.038305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:12.038813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:12.039248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:12.039920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:12.040371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:12.040790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:12.164306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:12.172439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8a58eb44-5b5d-4e61-8e4d-0b162d2567b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.145065 Sep 18 06:57:12.173003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1396/2464] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:12.838819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:12.839356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:12.846318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:12.846838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:12.847285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:12.847972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:12.848428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:12.848852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:12.985556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:12.993710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dd23895a-fbdc-4ae1-9e9e-6b3faf808b1b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.158845 Sep 18 06:57:12.994267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1397/2465] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:12 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:13.196617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:13.197298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:13.206887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:13.207613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:13.208271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:13.209310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:13.210006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:13.210692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:13.230771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:13.339327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:13.348035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-65b71458-5da7-4d42-a1f0-af8ccbbe5212 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.155686 Sep 18 06:57:13.348594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1398/2466] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:13 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:14.017349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:14.017952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:14.024455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:14.024929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:14.025357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:14.026025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:14.026476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:14.026917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:14.154631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:14.162718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ee3d209e-d2d5-4d81-9f11-ee7d5fea4e09 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149459 Sep 18 06:57:14.163282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1399/2467] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:14 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:14.371162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:14.371770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:14.377568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:14.378292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:14.378748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:14.379417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:14.379860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:14.380273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:14.511739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:14.519962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9f00ae4d-6045-42ff-bb94-83556eac5503 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152465 Sep 18 06:57:14.520520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1400/2468] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:15.186005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:15.186641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:15.193070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:15.193551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:15.193989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:15.194689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:15.195144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:15.195568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:15.319952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:15.328704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-16c1bce4-c39d-4468-b476-b1ab4f1b8d0c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147089 Sep 18 06:57:15.329277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1401/2469] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:15 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:15.544881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:15.545522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:15.548431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:15.552298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:15.552782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:15.553214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:15.553900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:15.554353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:15.554809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:15.685948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:15.694190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-226c2a1e-dd0f-4a19-a3fc-76615376a2fa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.153661 Sep 18 06:57:15.694788 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1402/2470] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:16.352172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:16.352806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:16.359255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:16.359725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:16.360154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:16.360826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:16.361322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:16.361812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:16.489782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:16.497963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2b1d3ec5-c295-46cb-84f3-b95c7879171b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149670 Sep 18 06:57:16.498523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1403/2471] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:16 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:16.718697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:16.719384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:16.725222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:16.725697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:16.726129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:16.726836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:16.727283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:16.727701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:16.856318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:16.864626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-45ae9f18-97b7-4377-bae3-d29c090ff21a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149782 Sep 18 06:57:16.865186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1404/2472] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:17.521547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:17.522160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:17.529167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:17.529674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:17.530109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:17.530886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:17.531337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:17.531760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:17.658829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:17.667290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7cbc8bf7-421c-4643-9e2a-6448eaa1acc4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149535 Sep 18 06:57:17.667898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1405/2473] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:17 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:17.888649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:17.889316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:17.895099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:17.895585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:17.896022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:17.896699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:17.897149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:17.897569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:18.024805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:18.033682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b5ca7e5c-da16-4687-bc84-b1b28674ffa2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149294 Sep 18 06:57:18.034249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1406/2474] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:18.691525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:18.692127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:18.698632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:18.699115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:18.699548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:18.700281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:18.700731 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:18.701155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:18.843541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:18.851915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-de5da4c1-8b81-4f33-ac0e-6ee3243f8352 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.164196 Sep 18 06:57:18.852485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1407/2475] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:18 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:19.002419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:19.003351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:19.009104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:19.009583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:19.010019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:19.010742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:19.011200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:19.011624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:19.036271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:57:19.110129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ea1c926d-70ac-43b8-93c8-9c8fb67541ce tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 204 len: 0 microversion: 2.1 time: 0.111259 Sep 18 06:57:19.110970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1408/2476] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:57:18 2022] DELETE /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 18 06:57:19.116114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:19.116799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:19.123302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:19.123780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:19.124210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:19.124889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:19.125330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:19.125746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:19.253911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:19.262129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-60e1c81a-0325-41de-b054-e513b08a6593 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149894 Sep 18 06:57:19.263250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1409/2477] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:19.267599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:19.268202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:19.273571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:19.274043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:19.274475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:19.275182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:19.275636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:19.276074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:19.405587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:19.413243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e4e10525-24da-42ab-8486-0b23eca345a2 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1680 microversion: 2.1 time: 0.149057 Sep 18 06:57:19.413808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1410/2478] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:19 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1680 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:19.875610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:19.876242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:19.883029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:19.883486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:19.884184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:19.884864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:19.885309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:19.885733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:20.010645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:20.019430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c8576180-2d0b-40f7-9228-578df754910a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147672 Sep 18 06:57:20.019999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1411/2479] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:19 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:20.285648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:20.286260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:20.292168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:20.292681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:20.293120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:20.293877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:20.294349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:20.294808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:20.421631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:20.429971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-09d1ebfc-c06f-41b6-a88b-58b91f9af57c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148073 Sep 18 06:57:20.430540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1412/2480] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:20 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:20.438862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:20.439449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:20.445118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:20.445598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:20.446030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:20.446746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:20.447204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:20.447621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:20.466812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:20.690917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:20.698482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1802410a-0105-4492-b732-812754861261 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1680 microversion: 2.1 time: 0.264016 Sep 18 06:57:20.699084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1413/2481] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:20 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1680 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:21.042238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:21.042951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:21.049330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:21.050176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:21.050632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:21.051324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:21.051774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:21.052198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:21.253029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:21.261231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d3f76259-ff9c-4b15-b497-1e2d8bc1d7f2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.222835 Sep 18 06:57:21.261802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1414/2482] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:21 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:21.408309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-89df97f9-0cb5-4b75-9e14-481b9365ab61 req-d9c9c956-872a-4fa1-8bef-492a47f1e98f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ceaad6c7-d3bd-45ed-9e68-010c68b41f1b", "name": "network-vif-deleted", "tag": "0a70bc98-af92-4192-933a-5024fbdfa5f0"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:57:21.420982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-89df97f9-0cb5-4b75-9e14-481b9365ab61 req-d9c9c956-872a-4fa1-8bef-492a47f1e98f service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:21.421465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-89df97f9-0cb5-4b75-9e14-481b9365ab61 req-d9c9c956-872a-4fa1-8bef-492a47f1e98f service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:21.421897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-89df97f9-0cb5-4b75-9e14-481b9365ab61 req-d9c9c956-872a-4fa1-8bef-492a47f1e98f service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:21.443296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-89df97f9-0cb5-4b75-9e14-481b9365ab61 req-d9c9c956-872a-4fa1-8bef-492a47f1e98f service nova] Creating event network-vif-deleted:0a70bc98-af92-4192-933a-5024fbdfa5f0 for instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b on hv-cinder-85535 Sep 18 06:57:21.449271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-89df97f9-0cb5-4b75-9e14-481b9365ab61 req-d9c9c956-872a-4fa1-8bef-492a47f1e98f service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044003 Sep 18 06:57:21.451093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1415/2483] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:57:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:57:21.459398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:21.460329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:21.469617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:21.470125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:21.470601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:21.471318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:21.471778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:21.472212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:21.633761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:21.642535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-adc0b150-af0e-49fd-8f47-8d0792d0ba50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.189975 Sep 18 06:57:21.643472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1416/2484] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:21.721401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:21.722388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:21.729982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:21.731111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:21.732072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:21.733503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:21.733819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:21.734540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:21.769243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:21.769717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:21.770142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:21.770604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lazy-loading 'fault' on Instance uuid ceaad6c7-d3bd-45ed-9e68-010c68b41f1b {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:57:21.820184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:21.828262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a8f286ce-364e-4816-a431-dc4e0094f310 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 200 len: 1330 microversion: 2.1 time: 0.110916 Sep 18 06:57:21.828935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1417/2485] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:21 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 1330 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:22.285243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:22.285880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:22.292357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:22.292848 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:22.293282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:22.294506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:22.294985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:22.295414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:22.427704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:22.437328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e379792e-5597-4d97-9a14-b4e590cc57d2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.156180 Sep 18 06:57:22.437887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1418/2486] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:22 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:22.666454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:22.667095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:22.673882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:22.674371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:22.674846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:22.675567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:22.676021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:22.676446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:22.801008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:22.809224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c23efa19-b91d-4e5d-83dd-7cb1fb5cd845 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146604 Sep 18 06:57:22.809782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1419/2487] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:22 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:22.851443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:22.852051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:22.858329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:22.858839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:22.859281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:22.859965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:22.860420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:22.860846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:22.874410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b HTTP/1.1 Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:57:22.876059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:57:22.878127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'ceaad6c7-d3bd-45ed-9e68-010c68b41f1b'} Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:57:22.879190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:57:22.880857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:57:22.882316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:57:22.882316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f70009fe-ec4c-4481-8d72-7b4f3de759c8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 404 len: 111 microversion: 2.1 time: 0.029559 Sep 18 06:57:22.882316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1420/2488] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:22 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 06:57:22.885802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a46fa991-2d24-4843-99e8-2e3c663c3edb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '.version_select at 0x7fe673cf8d30>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:22.904350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a46fa991-2d24-4843-99e8-2e3c663c3edb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-285570834" status: 202 len: 0 microversion: 2.1 time: 0.021906 Sep 18 06:57:22.904975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1421/2489] 10.222.0.20 () {58 vars in 1218 bytes} [Sun Sep 18 06:57:22 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-285570834 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:57:23.461514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:23.462242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:23.469396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:23.469985 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:23.470418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:23.471153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:23.471612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:23.472037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:23.597997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:23.606240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8f43d1da-72f2-4cc7-8524-271964f4a1ee tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149096 Sep 18 06:57:23.606832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1422/2490] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:23 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:23.611436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c15d660e-81d3-479a-9a1c-ce2be642154a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: .version_select at 0x7fe673bb7310>, body: {"keypair": {"name": "tempest-keypair-66806330", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMLI1b9mI9L92nSYZPOHAPtRZvpouTLbJcd6d3nQ6MJEmi/BHzvVM9gDO1yBqplYhZZT5mxuVOqnZZ4jCfYgo4w/nQElKvFQVouZlzbkdyhnsEYqd7hr4LdCEqLP5T30eA=="}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:57:23.626793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-c15d660e-81d3-479a-9a1c-ce2be642154a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:57:23.630864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-c15d660e-81d3-479a-9a1c-ce2be642154a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for user d85160b01d824bc3a32b425fab98d2c7 and project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'key_pairs'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:57:23.652007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c15d660e-81d3-479a-9a1c-ce2be642154a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.044033 Sep 18 06:57:23.652581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1423/2491] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 06:57:23 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:57:23.832288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:23.832931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:23.839067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:23.839525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:23.839960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:23.840701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:23.841157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:23.841585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:23.975892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:23.984717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5b69537d-473d-4e6d-873d-425d090e8a85 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.156398 Sep 18 06:57:23.985280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1424/2492] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:24.632031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:24.632689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:24.639087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:24.639568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:24.640001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:24.640712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:24.641142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:24.641562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:24.770096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:24.778341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-61331483-5cd1-4859-83d6-8816612d04c6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150521 Sep 18 06:57:24.778935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1425/2493] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:24 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:24.944164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-177604255", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d572beb8-e9a5-4e8d-a0f6-5ec49c88e496"}], "security_groups": [{"name": "tempest-securitygroup--1114202124"}], "key_name": "tempest-keypair-66806330", "user_data": "}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:57:25.049251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] validate_networks() for [('d572beb8-e9a5-4e8d-a0f6-5ec49c88e496', None, None, None, None, None)] {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 06:57:25.418004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 06:57:25.418443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 06:57:25.418957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 06:57:25.419370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 06:57:25.419787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 06:57:25.428576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:25.429057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:25.429490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:25.430213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:25.430699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:25.431127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:25.447469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 06:57:25.453926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Getting quotas for user d85160b01d824bc3a32b425fab98d2c7 and project 0c5d2a3f2d944529b8c505fa23b5a10f. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 06:57:25.463239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 06:57:25.466397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 06:57:25.515946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-297dc44e-b2d9-4c1a-b39a-006529eb8449 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.576238 Sep 18 06:57:25.516606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1426/2494] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 06:57:24 2022] POST /compute/v2.1/servers => generated 402 bytes in 578 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 06:57:25.522075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:25.522888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:25.529822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:25.530319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:25.530789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:25.531504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:25.531962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:25.532393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:25.665678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:25.674170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f9b27c6c-caa7-42ac-b3ac-e7284c1cc9d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.155809 Sep 18 06:57:25.674856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1427/2495] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:25.679446 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6f679928-28f5-4c62-8c14-079e49f27fdf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:25.680195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6f679928-28f5-4c62-8c14-079e49f27fdf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:25.732939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6f679928-28f5-4c62-8c14-079e49f27fdf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:25.739831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6f679928-28f5-4c62-8c14-079e49f27fdf tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1239 microversion: 2.1 time: 0.063917 Sep 18 06:57:25.740512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1428/2496] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:25 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1239 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:25.801482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:25.802697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:25.809895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:25.810528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:25.811093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:25.812041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:25.812597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:25.813262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:25.973549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:25.978123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'services' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:57:25.987122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7abd3c2d-c7c1-47ab-89f1-e938b879bc55 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.185252 Sep 18 06:57:25.989560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1429/2497] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:25 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:26.699567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:26.700811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:26.707443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:26.708024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:26.708507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:26.709200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:26.709657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:26.710082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:26.914254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:26.922637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-614fb141-4659-45e1-9767-49fec9f7c4df tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.227645 Sep 18 06:57:26.923166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1430/2498] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:26 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:26.927454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:26.928054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:26.933460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:26.933944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:26.934462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:26.935153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:26.935606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:26.936049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:27.005660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:27.013824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d713fd0f-0f48-4c86-8dd1-9a5984c1f28a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1297 microversion: 2.1 time: 0.089692 Sep 18 06:57:27.014394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1431/2499] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:26 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1297 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:27.018816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:27.019423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:27.025342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:27.025826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:27.026267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:27.026979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:27.027438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:27.028527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:27.489902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:27.498191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ea2645c2-d268-4a2d-b85e-0a036ddde1e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.482812 Sep 18 06:57:27.498796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1432/2500] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:27 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:27.945829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:27.947375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:27.952096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:27.958056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:27.958558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:27.959032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:27.959747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:27.960210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:27.960998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:28.192476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:28.201614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-183089f5-01ab-465f-94d0-3c537bfecbe0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.259981 Sep 18 06:57:28.202371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1433/2501] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:28.207017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:28.207632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:28.213281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:28.213763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:28.214203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:28.214917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:28.215563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:28.215992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:28.421993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:28.425524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:28.430857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dac0004a-dd3c-4f51-9b65-0e31ecffe17d tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1365 microversion: 2.1 time: 0.227348 Sep 18 06:57:28.431444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1434/2502] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:28 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1365 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:28.521770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:28.522371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:28.528294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:28.528774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:28.529210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:28.529891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:28.530341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:28.530806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:28.749600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:28.758243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d5c5afce-0369-4a7c-964a-660632d942bc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.240760 Sep 18 06:57:28.758842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1435/2503] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:28 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:28.845688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-8564f599-33f3-4a14-8f78-31c6e880e7dd req-0bd76d5b-b9ed-4aab-92ff-02120799530a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48431e56-3b9d-4dbe-b0ad-10c0c59f12ae", "tag": "5fbe700a-3ce0-47a3-b833-ae3c3099e861"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:57:28.855376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-8564f599-33f3-4a14-8f78-31c6e880e7dd req-0bd76d5b-b9ed-4aab-92ff-02120799530a service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:28.855800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-8564f599-33f3-4a14-8f78-31c6e880e7dd req-0bd76d5b-b9ed-4aab-92ff-02120799530a service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:28.856818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-8564f599-33f3-4a14-8f78-31c6e880e7dd req-0bd76d5b-b9ed-4aab-92ff-02120799530a service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:28.869418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-8564f599-33f3-4a14-8f78-31c6e880e7dd req-0bd76d5b-b9ed-4aab-92ff-02120799530a service nova] Creating event network-changed:5fbe700a-3ce0-47a3-b833-ae3c3099e861 for instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae on hv-cinder-85535 Sep 18 06:57:28.875866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-8564f599-33f3-4a14-8f78-31c6e880e7dd req-0bd76d5b-b9ed-4aab-92ff-02120799530a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036036 Sep 18 06:57:28.876477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1436/2504] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:57:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:57:29.226568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:29.227358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:29.233945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:29.234430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:29.234896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:29.235583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:29.236041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:29.236467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:29.367588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:29.375820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4fb1f05e-809f-4fe2-99e9-58579c711661 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154036 Sep 18 06:57:29.376383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1437/2505] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:29.453654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:29.454355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:29.460335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:29.460811 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:29.461265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:29.461941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:29.462383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:29.462830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:29.595499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:29.603047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc0a4443-b6e3-43fc-8b6a-b6a7d93b4386 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1365 microversion: 2.1 time: 0.153192 Sep 18 06:57:29.603605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1438/2506] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:29 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1365 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:29.780948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:29.781554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:29.787439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:29.787914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:29.788343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:29.789017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:29.789463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:29.789880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:29.917087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:29.925722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-355120fd-4691-4ef0-b0c4-64de984b39c8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148476 Sep 18 06:57:29.926286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1439/2507] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:29 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:30.399066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:30.399753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:30.406928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:30.407421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:30.407862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:30.408560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:30.409015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:30.409445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:30.596722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:30.604973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9219cef1-f50d-45fa-bb25-8ec2297e521e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.209804 Sep 18 06:57:30.605542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1440/2508] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:30 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:30.625461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:30.626092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:30.631990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:30.632467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:30.632896 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:30.633571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:30.634018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:30.634435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:30.760719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:30.768827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-270bd514-392f-422d-8a43-92a6d1113742 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1365 microversion: 2.1 time: 0.147603 Sep 18 06:57:30.769396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1441/2509] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:30 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1365 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:30.949120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:30.949727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:30.955588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:30.956059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:30.956490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:30.957163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:30.957615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:30.958032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:31.085219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:31.093938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-09bc5468-7073-492f-9bf2-451e98d333a6 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148626 Sep 18 06:57:31.094507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1442/2510] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:30 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:31.628316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:31.628999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:31.635347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:31.635821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:31.636250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:31.636920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:31.637944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:31.638362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:31.767736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:31.775918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-120c4c33-dd9c-4c6a-a4e7-b5a314d97727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151711 Sep 18 06:57:31.776485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1443/2511] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:31.790900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:31.791506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:31.797294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:31.797782 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:31.798241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:31.798935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:31.799393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:31.799819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:31.924099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:31.931769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34e996ac-b2b6-4d92-874c-97d9f7b7b242 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1365 microversion: 2.1 time: 0.144421 Sep 18 06:57:31.932338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1444/2512] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:31 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1365 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:32.117610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:32.118146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:32.124603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:32.125079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:32.125509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:32.126180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:32.126654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:32.127078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:32.255072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:32.263212 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d3a62cd7-b895-4fa1-8230-d8fc4c275a02 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149655 Sep 18 06:57:32.263774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1445/2513] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:32 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:32.799292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:32.799945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:32.806420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:32.806929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:32.807364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:32.808041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:32.808487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:32.810646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:32.938676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:32.946814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b12701fb-0ba4-4c75-835c-d021734d5eac tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151426 Sep 18 06:57:32.947363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1446/2514] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:32 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:32.954856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:32.955474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:32.961167 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:32.961649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:32.962087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:32.962832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:32.963287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:32.963717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:33.091411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:33.098958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ed1a2eff-eeda-4cde-99fb-14a32d6144a5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1365 microversion: 2.1 time: 0.147578 Sep 18 06:57:33.099527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1447/2515] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:32 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1365 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:33.286117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:33.286754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:33.292584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:33.293062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:33.293485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:33.294163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:33.294624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:33.295049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:33.422955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:33.431874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5120632b-fa36-4155-b020-4ab688040cc3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149559 Sep 18 06:57:33.432457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1448/2516] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:33 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:33.971203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:33.971859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:33.978106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:33.978600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:33.979106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:33.979780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:33.980227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:33.981289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:34.107390 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:34.115563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8091ff06-986b-4424-9274-0c432e7a8070 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148324 Sep 18 06:57:34.116253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1449/2517] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:33 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:34.120935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:34.121576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:34.127209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:34.127950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:34.128389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:34.129074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:34.129529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:34.129960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:34.261087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:34.269246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f2d64230-30a3-42d4-ac87-c30ce81e0acb tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1365 microversion: 2.1 time: 0.151867 Sep 18 06:57:34.269810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1450/2518] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:34 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1365 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:34.456238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:34.456846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:34.463131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:34.463609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:34.464038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:34.464724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:34.465749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:34.466410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:34.592789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:34.600964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5511015e-4d23-430c-a2ea-bfd425e80a92 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148559 Sep 18 06:57:34.601558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1451/2519] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:34 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:35.138512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:35.139127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:35.145484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:35.146234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:35.146717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:35.147404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:35.147863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:35.148290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:35.152287 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:35.277599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:35.286088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-24ecc163-e66b-49ea-8459-3b8cdc26d97e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151464 Sep 18 06:57:35.286903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1452/2520] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:35.291931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:35.292550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:35.299018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:35.299504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:35.299938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:35.300633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:35.301084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:35.301505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:35.434567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:35.438937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:35.443530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-301e4c79-f3a5-4b60-bcba-4320ceb6695a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1365 microversion: 2.1 time: 0.155168 Sep 18 06:57:35.444099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1453/2521] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:35 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1365 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:35.448546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-0b705fb5-5e0c-4f80-90e6-eeffbc8803a3 req-d2936deb-b379-4ccc-a3c2-73a215f44b7f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48431e56-3b9d-4dbe-b0ad-10c0c59f12ae", "name": "network-vif-plugged", "status": "completed", "tag": "5fbe700a-3ce0-47a3-b833-ae3c3099e861"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:57:35.459191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-0b705fb5-5e0c-4f80-90e6-eeffbc8803a3 req-d2936deb-b379-4ccc-a3c2-73a215f44b7f service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:35.459676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-0b705fb5-5e0c-4f80-90e6-eeffbc8803a3 req-d2936deb-b379-4ccc-a3c2-73a215f44b7f service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:35.460113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-0b705fb5-5e0c-4f80-90e6-eeffbc8803a3 req-d2936deb-b379-4ccc-a3c2-73a215f44b7f service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:35.472854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-0b705fb5-5e0c-4f80-90e6-eeffbc8803a3 req-d2936deb-b379-4ccc-a3c2-73a215f44b7f service nova] Creating event network-vif-plugged:5fbe700a-3ce0-47a3-b833-ae3c3099e861 for instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae on hv-cinder-85535 Sep 18 06:57:35.479265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-0b705fb5-5e0c-4f80-90e6-eeffbc8803a3 req-d2936deb-b379-4ccc-a3c2-73a215f44b7f service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033661 Sep 18 06:57:35.479871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1454/2522] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:57:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:57:35.624841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:35.625656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:35.632490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:35.633043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:35.633485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:35.634180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:35.634655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:35.635083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:35.740467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=12 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:35.788871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:35.797114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1676ba22-3e3f-40b8-a1ee-81f6b7a4098b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.176586 Sep 18 06:57:35.797681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1455/2523] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:35 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:35.996981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=4 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:36.311740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:36.312290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:36.318720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:36.319201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:36.319641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:36.320327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:36.320780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:36.321205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:36.457505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:36.466824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-60df3431-c1cc-420d-a3c6-a499412b75c6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.159587 Sep 18 06:57:36.467407 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1456/2524] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:36.472518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:36.473115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:36.478845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:36.479323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:36.479876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:36.481065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:36.481513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:36.481924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:36.615133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:36.623319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7c94dda8-2da9-4bb7-9e87-f51531337c3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1365 microversion: 2.1 time: 0.154922 Sep 18 06:57:36.623878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1457/2525] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:36 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1365 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:36.820724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:36.821248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:36.827480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:36.827893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:36.828352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:36.829070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:36.829524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:36.829948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:36.961302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:36.970069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8c18c0d8-1fc1-4120-bdfa-e5d688b0a6a8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.153339 Sep 18 06:57:36.970643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1458/2526] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:36 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:37.490405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:37.491055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:37.497246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:37.497740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:37.498169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:37.498892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:37.499347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:37.499772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:37.631694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:37.640763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8d5cdf7d-df0a-4de8-afa7-5e8268022204 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154255 Sep 18 06:57:37.641322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1459/2527] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:37 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:37.645759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:37.646305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:37.653107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:37.653697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:37.654140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:37.654983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:37.655483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:37.655909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:37.782161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:37.789736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3f8aee6c-dbbd-46da-b9a9-64200227aeb7 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1556 microversion: 2.1 time: 0.147476 Sep 18 06:57:37.790299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1460/2528] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:37 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1556 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:37.811545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:37.812031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:37.817532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:37.818007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:37.818441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:37.819143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:37.819594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:37.820015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:37.896413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-96865f7c-ab28-47da-afc9-a9ccb15ae136 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.088785 Sep 18 06:57:37.897007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1461/2529] 10.222.0.20 () {58 vars in 1275 bytes} [Sun Sep 18 06:57:37 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae/os-interface => generated 285 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:57:37.993501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:37.994040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:38.003411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:38.003935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:38.004425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:38.005181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:38.005637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:38.006061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:38.228968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:38.237718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1bf7a8bf-8c02-4eb9-9f88-3fbf43977aad tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.248337 Sep 18 06:57:38.238291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1462/2530] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:37 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:38.663939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:38.664596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:38.671735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:38.672228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:38.672672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:38.673460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:38.673943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:38.674438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:39.141564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:39.149760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3b541dda-1a64-482d-9dd6-d0eee1e90534 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.489663 Sep 18 06:57:39.150340 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1463/2531] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:38 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:39.236422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-36d8f0f0-0b4a-4b29-9b8b-f4f19782cafe req-a74a3a62-0b29-42dc-9570-6837646e0bdd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48431e56-3b9d-4dbe-b0ad-10c0c59f12ae", "tag": "5fbe700a-3ce0-47a3-b833-ae3c3099e861"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:57:39.245672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-36d8f0f0-0b4a-4b29-9b8b-f4f19782cafe req-a74a3a62-0b29-42dc-9570-6837646e0bdd service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:39.246658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-36d8f0f0-0b4a-4b29-9b8b-f4f19782cafe req-a74a3a62-0b29-42dc-9570-6837646e0bdd service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:39.247087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-36d8f0f0-0b4a-4b29-9b8b-f4f19782cafe req-a74a3a62-0b29-42dc-9570-6837646e0bdd service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:39.258237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-36d8f0f0-0b4a-4b29-9b8b-f4f19782cafe req-a74a3a62-0b29-42dc-9570-6837646e0bdd service nova] Creating event network-changed:5fbe700a-3ce0-47a3-b833-ae3c3099e861 for instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae on hv-cinder-85535 Sep 18 06:57:39.264082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-36d8f0f0-0b4a-4b29-9b8b-f4f19782cafe req-a74a3a62-0b29-42dc-9570-6837646e0bdd service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031232 Sep 18 06:57:39.264677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1464/2532] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:57:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 06:57:39.269724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:39.270420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:39.276749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:39.277321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:39.277761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:39.278448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:39.278980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:39.279355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:39.490044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:39.498267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0528809e-990c-46f5-b8ee-9bba394ac04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.232395 Sep 18 06:57:39.505790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1465/2533] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:39 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:40.172417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:40.173031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:40.180000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:40.180489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:40.180921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:40.181613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:40.182058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:40.182478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:40.317649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:40.325959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-43b597ff-d238-4e68-a74f-cf81c3555409 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.157333 Sep 18 06:57:40.326534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1466/2534] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:40.521577 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:40.522450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:40.528492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:40.528972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:40.529476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:40.530521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:40.531234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:40.531870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:40.667614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:40.676315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-57e7840c-9a31-4f11-9f98-820ef590f760 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.158673 Sep 18 06:57:40.676874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1467/2535] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:40 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:41.348199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:41.348852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:41.355284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:41.355758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:41.356192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:41.356869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:41.357318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:41.357748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:41.486814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:41.495080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3046b019-7547-48da-80e3-50d202bba3e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150704 Sep 18 06:57:41.495643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1468/2536] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:41.699908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:41.700524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:41.706491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:41.707008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:41.707447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:41.708133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:41.708587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:41.709011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:41.835929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:41.844189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f22b626f-47f8-4244-ae8a-25e6dd9ee895 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148748 Sep 18 06:57:41.844764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1469/2537] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:41 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:42.519033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:42.519656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:42.526049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:42.526524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:42.527649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:42.528331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:42.528775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:42.529194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:42.546817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:42.659566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:42.667742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8aa5954b-5a55-4cfd-83ef-d2227c75a571 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152575 Sep 18 06:57:42.668301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1470/2538] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:42.867761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:42.868305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:42.874741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:42.875225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:42.875666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:42.876355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:42.876802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:42.877234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:43.005543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:43.014513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-67fdc6a0-6603-4951-a2db-3a275fa2e9c7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150705 Sep 18 06:57:43.015105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1471/2539] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:42 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:43.691058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:43.691703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:43.698008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:43.698481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:43.698941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:43.699616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:43.700060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:43.700477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:43.825188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:43.833289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-72c041f4-7cab-4358-ac27-3b98938502a5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146147 Sep 18 06:57:43.833850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1472/2540] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:44.037564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:44.038083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:44.044115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:44.044600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:44.045040 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:44.045718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:44.046173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:44.046620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:44.174489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:44.182719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6da6c8ee-7b3b-41cd-a1c7-f98ba1fb3094 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149103 Sep 18 06:57:44.183290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1473/2541] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:44 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:44.856933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:44.857581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:44.863868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:44.864352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:44.864830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:44.865511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:44.865962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:44.866383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:44.992175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:45.001482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c7eb6abf-306b-4828-92e5-ba25bb432e33 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148474 Sep 18 06:57:45.002052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1474/2542] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:45.205789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:45.206387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:45.212225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:45.212689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:45.213114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:45.213785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:45.214231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:45.214671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:45.343027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:45.351315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cee402b3-4cea-45b3-b391-a2067265f363 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149177 Sep 18 06:57:45.351902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1475/2543] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:45 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:46.025439 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:46.026078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:46.032975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:46.033454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:46.033886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:46.034557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:46.035029 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:46.035448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:46.161241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:46.169919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4b74a7db-ae4a-4529-a366-b6739a6ba624 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148281 Sep 18 06:57:46.170477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1476/2544] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:46.374894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:46.375518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:46.381751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:46.382232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:46.382710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:46.383389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:46.383833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:46.384253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:46.514903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:46.523007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-45a2c03b-7b99-4bd8-92ea-498e0b8d7103 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151864 Sep 18 06:57:46.523569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1477/2545] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:46 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:47.192769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:47.193403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:47.199821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:47.200301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:47.200732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:47.201410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:47.201859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:47.202280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:47.326910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:47.335026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-78dd639f-077e-4508-af8f-da06455ce3b1 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146095 Sep 18 06:57:47.335581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1478/2546] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:47.546947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:47.547590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:47.553514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:47.553994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:47.554428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:47.555144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:47.555592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:47.556011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:47.681915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:47.690050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c23259a4-0b39-46d5-95c8-f243356a1484 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147600 Sep 18 06:57:47.690630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1479/2547] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:47 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:48.358481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:48.359165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:48.365561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:48.366044 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:48.366506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:48.367215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:48.367669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:48.368091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:48.504520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:48.512741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c41aafb3-e233-4912-becb-1406c18b2a50 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.158205 Sep 18 06:57:48.513308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1480/2548] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:48.713966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:48.714680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:48.720883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:48.721366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:48.721800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:48.722479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:48.722955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:48.723380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:48.847818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:48.856692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-12975eae-6e8d-4040-9663-805fe2f97a1a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146502 Sep 18 06:57:48.857263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1481/2549] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:48 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:49.536196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:49.536843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:49.543350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:49.543844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:49.544286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:49.544989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:49.545441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:49.545864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:49.681170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:49.689443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0a505a07-4ca6-4ac6-ac7c-a649f717a03a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.157200 Sep 18 06:57:49.690005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1482/2550] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:49.879901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:49.880510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:49.883777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:49.888041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:49.888522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:49.888957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:49.889644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:49.890095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:49.890520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:50.018376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:50.027224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-20513048-988c-401d-a057-2fb76a0819f3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150654 Sep 18 06:57:50.027769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1483/2551] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:49 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:50.712755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:50.713413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:50.719221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:50.719718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:50.720149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:50.720826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:50.721274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:50.721692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:50.846605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:50.855315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-943b4259-2c21-493d-828d-0900ce0da45e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.145947 Sep 18 06:57:50.855892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1484/2552] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:51.049180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:51.049717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:51.056103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:51.056583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:51.057017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:51.057703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:51.058785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:51.059215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:51.189162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:51.197300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e1c5c041-de19-4671-8940-ed70958661a9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151934 Sep 18 06:57:51.197865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1485/2553] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:51 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:51.879323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:51.879971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:51.885766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:51.886891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:51.887329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:51.888008 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:51.888457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:51.888878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:52.014903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:52.022950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-730d4814-b10f-4302-a814-0f2ad1a108d8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147917 Sep 18 06:57:52.023513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1486/2554] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:52.220018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:52.220625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:52.226929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:52.227413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:52.227849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:52.228530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:52.228979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:52.229400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:52.233996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:52.360511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:52.368880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-79c82926-3a19-4537-851f-e528eb106e36 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152547 Sep 18 06:57:52.369440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1487/2555] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:52 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:53.045418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:53.046120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:53.052148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:53.052628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:53.053064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:53.053743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:53.054193 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:53.054652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:53.179931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:53.188070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9a2ec481-d7ef-4bfa-aeb8-787229661920 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146736 Sep 18 06:57:53.188643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1488/2556] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:53.392849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:53.393391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:53.399998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:53.400481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:53.400919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:53.401607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:53.402061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:53.402487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:53.534142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:53.542283 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1d0edebb-dffb-4842-b3ec-1a198725f8b8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.153269 Sep 18 06:57:53.542897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1489/2557] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:53 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:54.212020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:54.212770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:54.218770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:54.219255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:54.219690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:54.220375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:54.220825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:54.221253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:54.349982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:54.358113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-daf5c8fc-9235-4ef3-8dda-175053783396 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149987 Sep 18 06:57:54.358730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1490/2558] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:54.566413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:54.567006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:54.573375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:54.573854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:54.574293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:54.575006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:54.575493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:54.575916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:54.701943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:54.710097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-19d3a677-be62-40a3-a771-6f030f27333e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147658 Sep 18 06:57:54.710688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1491/2559] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:54 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:57:55.383422 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:55.384102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:55.390080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:55.390561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:55.391025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:55.391702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:55.392153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:55.392578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:55.521835 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:55.530023 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ca81d02a-10ac-4cfa-ae39-6f4a23ea138e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150503 Sep 18 06:57:55.530610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1492/2560] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:55.733477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:55.734088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:55.740516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:55.740995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:55.741430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:55.742110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:55.742558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:55.743012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:55.870831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:55.881295 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dfe0b4ad-d0fe-4567-957b-0a018c5f9ee4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151837 Sep 18 06:57:55.882365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1493/2561] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:55 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:56.553574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:56.554201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:56.561694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:56.562339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:56.563149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:56.564346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:56.565031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:56.565469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:56.695403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:56.704117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6ef20885-ac2e-4607-8cb6-45757fcbdfcc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154421 Sep 18 06:57:56.704686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1494/2562] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:56 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:56.906431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:56.907072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:56.913362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:56.913833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:56.914263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:56.914991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:56.915436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:56.915849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:57.040655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:57.049238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bf4fbb72-d165-4e24-8b4a-249f7d7f3ce2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146539 Sep 18 06:57:57.049798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1495/2563] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:56 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:57.726809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:57.727458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:57.733460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:57.733937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:57.734371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:57.735081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:57.735528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:57.735948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:57.862647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:57.870947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1e6dc9bf-f48e-4607-8d5a-1f72a8128658 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148053 Sep 18 06:57:57.872047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1496/2564] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:58.073160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:58.073709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:58.080632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:58.081107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:58.081538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:58.082215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:58.082693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:58.083114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:58.208020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:58.216106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0faec9f-d8ea-4c75-bd62-ce99acddc0cb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146815 Sep 18 06:57:58.216668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1497/2565] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:58 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:58.895671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:58.896314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:58.902245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:58.903244 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:58.903675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:58.904355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:58.904800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:58.905219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:59.045120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:59.053949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-613f968a-f8ff-44e5-b881-492282c44663 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.162166 Sep 18 06:57:59.054612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1498/2566] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:58 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:57:59.239995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:57:59.240635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:57:59.247243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:59.247724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:59.248182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:59.248853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:57:59.249304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:57:59.249728 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:57:59.386887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:57:59.395368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-31b2844f-b3fc-48e4-a431-a368462fc43d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.159235 Sep 18 06:57:59.395926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1499/2567] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:57:59 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:00.077181 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:00.077875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:00.084065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:00.084530 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:00.084995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:00.085679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:00.086133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:00.086559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:00.212311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:00.221196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-077647e6-eafe-4c43-a94f-d1f441243f9e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147954 Sep 18 06:58:00.221768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1500/2568] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:00.418868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:00.419476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:00.425809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:00.426284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:00.426751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:00.427428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:00.427899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:00.428318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:00.558814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:00.567089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a4eca03f-e3c0-43f0-9ebe-c1e0c6fe328e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152584 Sep 18 06:58:00.567664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1501/2569] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:00 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:01.245280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:01.245958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:01.252790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:01.253272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:01.253709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:01.254393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:01.254876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:01.255301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:01.381935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:01.390198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-94680ca2-e519-4d74-b053-b33ed09670b4 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148798 Sep 18 06:58:01.390791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1502/2570] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:01.591529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:01.592100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:01.598555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:01.599073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:01.599506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:01.600188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:01.600636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:01.601060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:01.727965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:01.732121 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:58:01.737900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fb82dd62-a33b-44d7-835f-6ab364e17577 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151002 Sep 18 06:58:01.738472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1503/2571] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:01 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:02.413605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:02.414319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:02.420424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:02.420902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:02.421339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:02.422021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:02.422472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:02.422920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:02.552460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:02.561191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1757b19d-1dd0-403a-be0d-c0c5df5950db tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151452 Sep 18 06:58:02.561759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1504/2572] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:02.761128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:02.761712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:02.768288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:02.768763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:02.769198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:02.770064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:02.770541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:02.770991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:02.899876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:02.908192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-645c527f-5423-49da-b8fe-70f63cb831ab tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150954 Sep 18 06:58:02.908758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1505/2573] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:02 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:03.584666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:03.585290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:03.591948 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:03.592436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:03.592878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:03.593567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:03.594022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:03.594449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:03.727885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:03.736418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6eb8b406-a65d-4b55-abb7-5a792175959e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.155655 Sep 18 06:58:03.736979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1506/2574] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:03.931712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:03.932547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:03.939303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:03.939780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:03.940208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:03.940883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:03.941331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:03.941750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:04.070708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:04.079401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2786da01-c3db-40a6-819c-9483c3f305c2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151413 Sep 18 06:58:04.079969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1507/2575] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:03 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:04.759443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:04.760094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:04.765940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:04.766418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:04.766885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:04.767562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:04.768010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:04.768432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:04.896892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:04.905494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-33a58b58-e06b-4584-a207-ac44b0f7f372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149980 Sep 18 06:58:04.906049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1508/2576] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:04 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:05.102649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:05.103185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:05.109598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:05.110082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:05.110521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:05.111232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:05.111686 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:05.112111 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:05.237026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:05.245178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4585dc57-c206-443f-8ef4-f733c5cee980 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146677 Sep 18 06:58:05.245748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1509/2577] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:05 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:05.929057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:05.929724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:05.936545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:05.937054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:05.937998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:05.939083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:05.939546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:05.939979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:06.066802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:06.074936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e39fc839-9d81-48d4-bf38-d345397cb271 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149725 Sep 18 06:58:06.075502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1510/2578] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:06.268430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:06.268957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:06.275294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:06.275747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:06.276173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:06.276847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:06.277805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:06.278222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:06.409449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:06.417665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-20ff1f80-5d90-4604-9ca4-abb30d61833e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152992 Sep 18 06:58:06.418933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1511/2579] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:06 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:07.098398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:07.099143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:07.105172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:07.105738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:07.106173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:07.106886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:07.107338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:07.107763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:07.234539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:07.242743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e6512cef-7caf-4b29-a084-c0868ae5f995 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148201 Sep 18 06:58:07.243303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1512/2580] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:07.441177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:07.441796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:07.448269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:07.448806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:07.449228 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:07.449917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:07.450376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:07.450833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:07.579563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:07.587681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6fc833fc-4da7-4159-abe2-10a9e9829f21 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150350 Sep 18 06:58:07.588250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1513/2581] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:07 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:08.267114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:08.267802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:08.273687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:08.274171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:08.274617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:08.275558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:08.276011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:08.276438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:08.411116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:08.419335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-06c04c9e-6f31-4953-a519-9f88322ce335 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.156094 Sep 18 06:58:08.419898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1514/2582] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:08.611420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:08.611955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:08.618237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:08.618750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:08.619182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:08.619855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:08.620304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:08.621249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:08.747292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:08.755662 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-96a9adbe-5a82-4773-ab7b-ccb76a791a72 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148177 Sep 18 06:58:08.756220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1515/2583] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:08 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:09.443002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:09.443640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:09.450471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:09.450990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:09.451712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:09.452410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:09.452871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:09.453298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:09.583149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:09.592267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-27d579c1-93d0-4664-9576-7148976e3a12 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.153190 Sep 18 06:58:09.593020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1516/2584] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:09.597591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:09.598081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:09.603581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:09.604062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:09.604507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:09.605391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:09.605850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:09.606281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:09.630802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f65088d2-e66d-4eec-b61b-965e09b20265 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae/ips" status: 200 len: 151 microversion: 2.1 time: 0.036488 Sep 18 06:58:09.631578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1517/2585] 10.222.0.20 () {58 vars in 1248 bytes} [Sun Sep 18 06:58:09 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae/ips => generated 151 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:58:09.779343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:09.779954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:09.785812 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:09.786291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:09.786827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:09.787509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:09.787969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:09.788394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:09.913182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:09.921450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a17a5db2-caba-409f-a6f9-9a08b2bb7b93 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146491 Sep 18 06:58:09.922020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1518/2586] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:09 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:10.614946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:10.615607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:10.621997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:10.622471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:10.622934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:10.623616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:10.624060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:10.624487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:10.757164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:10.765358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8bacdbe2-a4ac-4fc5-8ff8-a9c1557faf6e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154295 Sep 18 06:58:10.765923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1519/2587] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:10.945043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:10.945650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:10.951983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:10.952456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:10.952885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:10.953558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:10.954003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:10.954419 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:11.082264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:11.085633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:58:11.091754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8ce2be3a-068b-4a7b-95cc-dfaa98ed8aae tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150478 Sep 18 06:58:11.092318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1520/2588] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:10 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:11.788901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:11.789536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:11.796499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:11.796975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:11.797413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:11.798094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:11.798547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:11.798994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:11.927488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:11.936292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5289ead9-0894-445e-9032-0620700392e2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151456 Sep 18 06:58:11.936863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1521/2589] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:12.114271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:12.114905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:12.121278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:12.121843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:12.122353 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:12.123171 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:12.123715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:12.124329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:12.282224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:12.291562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-32d86ccf-fa65-4253-9e98-74ca5a4e8ae7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.180971 Sep 18 06:58:12.294749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1522/2590] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:12 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:12.959050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:12.959678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:12.966070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:12.966554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:12.967015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:12.967697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:12.968148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:12.968571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:13.093712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:13.101918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-88c23fe4-21af-40cb-9af1-10e84ea1fcae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146761 Sep 18 06:58:13.102491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1523/2591] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:12 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:13.315046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:13.315584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:13.321441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:13.321919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:13.322355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:13.323060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:13.323513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:13.323936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:13.454431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:13.462798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-21a8bb9a-37b9-4f9a-b014-34366b37f6c1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151392 Sep 18 06:58:13.463368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1524/2592] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:13 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:14.125701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:14.126426 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:14.133264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:14.133750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:14.134216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:14.134924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:14.135376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:14.135798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:14.262057 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:14.270455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4a3bfcd2-52b2-4a2b-b9b9-0490ebe2410f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148600 Sep 18 06:58:14.271047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1525/2593] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:14.485224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:14.485845 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:14.492347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:14.492856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:14.493296 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:14.493992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:14.494450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:14.494905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:14.622475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:14.631206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-da5f2ed4-43e3-476e-98e6-637a148a1c59 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149792 Sep 18 06:58:14.631798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1526/2594] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:14 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:15.294899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:15.295588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:15.301817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:15.302292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:15.302758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:15.303436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:15.303880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:15.304300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:15.429737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:15.437925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b003750c-4e4a-45e7-b472-8c009e1d3eae tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147582 Sep 18 06:58:15.438484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1527/2595] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:15.658913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:15.659523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:15.665388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:15.665862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:15.666292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:15.666989 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:15.667482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:15.667900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:15.794517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:15.802889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-98c4ecfd-0a07-4538-8e09-4bb869b8f592 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147700 Sep 18 06:58:15.803462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1528/2596] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:15 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:16.461220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:16.461842 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:16.468359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:16.468839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:16.469274 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:16.469955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:16.470406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:16.470857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:16.607505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:16.615775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6d16a6ac-fffe-4af3-b8c0-66e114306b90 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.158396 Sep 18 06:58:16.616338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1529/2597] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:16.826546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:16.827222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:16.833197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:16.833679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:16.834114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:16.835016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:16.835279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:16.835704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:16.963063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:16.971734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0de09d93-f4d3-451a-8cf0-a032c307af17 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149125 Sep 18 06:58:16.972293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1530/2598] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:16 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:17.640747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:17.641424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:17.647859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:17.648341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:17.648769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:17.649451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:17.649899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:17.650320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:17.779778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:17.787877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-032cc9be-ea63-48c2-8b88-9ee05ebd4a38 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150944 Sep 18 06:58:17.788438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1531/2599] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:17.993994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:17.994523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:18.000846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:18.001323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:18.001753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:18.002714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:18.003165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:18.003623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:18.285003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:18.293706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-48099cad-710d-4de7-beeb-99bc770b0156 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.303427 Sep 18 06:58:18.294262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1532/2600] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:17 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:18.811713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:18.812397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:18.815676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:58:18.819427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:18.819902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:18.820334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:18.821004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:18.821455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:18.821869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:18.952017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:18.960120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-78f6b2b1-0812-41a3-a5e1-6a25f6653a8a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152750 Sep 18 06:58:18.960667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1533/2601] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:19.315449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:19.316065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:19.321892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:19.322373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:19.322851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:19.323558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:19.324011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:19.324435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:19.449391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:19.457735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cee233e0-be64-4177-a561-08833e1a5eb8 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146017 Sep 18 06:58:19.458285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1534/2602] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:19 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:19.983051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:19.983724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:19.990068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:19.990551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:19.991027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:19.991710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:19.992163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:19.992586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:20.115868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:20.124000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-46c6f82e-5534-46ef-bd7e-14761aff0057 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.144747 Sep 18 06:58:20.124555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1535/2603] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:19 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:20.481317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:20.481936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:20.487972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:20.488457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:20.488898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:20.489586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:20.490043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:20.490469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:20.623007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:20.631569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-69548818-df4d-47a8-a8dd-efa2e118f786 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154104 Sep 18 06:58:20.632139 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1536/2604] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:20 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:21.147847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:21.148572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:21.154917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:21.155401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:21.155843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:21.156524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:21.156978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:21.157401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:21.174943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:58:21.282425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:21.291062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b7c29a45-dc72-436b-a213-7cf866f1e7d5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147005 Sep 18 06:58:21.291652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1537/2605] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:21.655490 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:21.656101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:21.662136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:21.662642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:21.663079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:21.663761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:21.664213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:21.664636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:21.796520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:21.804720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f8efc454-dea0-4367-adf5-69c60bea3e1e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.153105 Sep 18 06:58:21.805289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1538/2606] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:21 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:22.314629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:22.315236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:22.321647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:22.322131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:22.322598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:22.323793 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:22.324252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:22.324681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:22.452273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:22.460605 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-535a7186-3a2f-4b44-9d78-3a818cdba55b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149822 Sep 18 06:58:22.461164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1539/2607] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:22 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:22.829134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:22.829660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:22.835452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:22.835928 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:22.836357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:22.837041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:22.837470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:22.837884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:22.964136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:22.972326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-462f870b-b7e4-4e91-8c44-0b023c0e13db tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147131 Sep 18 06:58:22.972904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1540/2608] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:22 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:23.485404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:23.486019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:23.492518 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:23.493002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:23.493447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:23.494142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:23.494621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:23.495053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:23.623112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:23.631565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-884039bc-2c4f-4820-a9ad-7325dee80d00 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150085 Sep 18 06:58:23.632147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1541/2609] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:23.995559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:23.996177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:24.002347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:24.002860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:24.003301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:24.003986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:24.004442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:24.004868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:24.129182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:24.137976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c734c51c-ec84-4129-bdbb-9b6acda6e431 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146219 Sep 18 06:58:24.138479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1542/2610] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:23 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:24.655172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:24.655742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:24.662594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:24.663082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:24.663512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:24.664220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:24.664668 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:24.665090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:24.812302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:24.820526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-88b13278-95b6-47ab-b0d9-71228de7e22c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.169772 Sep 18 06:58:24.821087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1543/2611] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:24 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:25.160924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:25.161534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:25.168031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:25.168504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:25.168934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:25.169607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:25.170054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:25.170474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:25.295130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:25.303213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3bd45809-fb18-48cd-aac0-b91382317364 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146233 Sep 18 06:58:25.303775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1544/2612] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:25 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:25.843990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:25.844606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:25.851103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:25.851608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:25.852032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:25.852718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:25.853175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:25.853597 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:25.986239 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:25.995707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a50b653e-0166-45b4-8d78-1a6646a057aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154818 Sep 18 06:58:25.996650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1545/2613] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:58:26.327867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:26.328827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:26.334872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:26.335357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:26.335797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:26.336497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:26.336952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:26.337382 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:26.465799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:26.474003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-65fc6ac1-b13c-48e6-9589-a83e39c2a7a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150988 Sep 18 06:58:26.474563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1546/2614] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:26 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:58:27.019761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:27.020449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:27.027371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:27.027858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:27.028300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:27.028987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:27.029444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:27.029893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:27.155277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:27.163538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b88720ea-ea20-491f-a91d-819e31254727 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147679 Sep 18 06:58:27.164120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1547/2615] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:27 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:27.497911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:27.498523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:27.504980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:27.505465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:27.505908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:27.506619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:27.507079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:27.507507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:27.635404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:27.643555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3ac041ab-b15c-4b8e-8917-0f9e9d73a04a tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150186 Sep 18 06:58:27.644125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1548/2616] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:27 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:28.186603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:28.187303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:28.193772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:28.194261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:28.194735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:28.195429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:28.195884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:28.196315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:28.321592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:28.330199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cd9bec25-ed48-4983-a8bd-4d42226989b3 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147441 Sep 18 06:58:28.330797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1549/2617] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:28 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:28.667716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:28.668329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:28.674307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:28.674826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:28.675265 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:28.675953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:28.676405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:28.676828 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:28.807927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:28.816123 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7401a50b-d6c4-4224-84e0-de97e457110e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.152338 Sep 18 06:58:28.816689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1550/2618] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:28 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:29.354600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:29.355299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:29.361651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:29.362133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:29.362593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:29.363286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:29.363747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:29.364175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:29.498918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:29.507384 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5917772c-05cf-4a5d-bc97-59f94b384d7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.157309 Sep 18 06:58:29.507953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1551/2619] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:29 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:29.839849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:29.840453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:29.846553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:29.847070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:29.847506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:29.848194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:29.848650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:29.849105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:29.974019 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:29.982769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2225a76b-39af-4977-87dd-6b32726d3d42 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146739 Sep 18 06:58:29.983337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1552/2620] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:29 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:30.530760 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:30.531376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:30.537770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:30.538251 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:30.538716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:30.539400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:30.539853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:30.540276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:30.684975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:30.693120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1e19db2b-7492-48ee-83ba-a3867edb13b2 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.166908 Sep 18 06:58:30.693685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1553/2621] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:30 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:31.008611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:31.009215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:31.015850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:31.016328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:31.016756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:31.017432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:31.017878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:31.018297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:31.022202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:58:31.144445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:31.152562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-351ce5c9-a8f6-45f6-99c7-4b73c083d23d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147818 Sep 18 06:58:31.153127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1554/2622] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:31 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:31.718954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:31.719606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:31.726133 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:31.726639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:31.727130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:31.728180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:31.728820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:31.729874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:32.115937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:32.124176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-25f6b194-2e80-4722-9d35-bbe8cf528a83 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.409231 Sep 18 06:58:32.124743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1555/2623] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:31 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:32.174951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:32.175559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:32.181071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:32.181556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:32.181997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:32.182713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:32.183178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:32.183610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:32.308746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:32.317558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f63e1ccd-5f7d-432e-9ddc-aa943b9bc593 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146068 Sep 18 06:58:32.318124 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1556/2624] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:32 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:33.148442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:33.149275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:33.155957 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:33.156437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:33.156873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:33.157559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:33.158013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:33.158440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:33.289581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:33.298466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5e1ada79-092e-4140-b117-d102c02d1db0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.153837 Sep 18 06:58:33.299105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1557/2625] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:33 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:33.342084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:33.342706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:33.348175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:33.348648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:33.349079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:33.349750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:33.350196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:33.350636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:33.478025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:33.486191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a7c50b89-cb37-495c-8359-9bf5044a46e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148108 Sep 18 06:58:33.487394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1558/2626] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:33 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:34.321564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:34.322248 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:34.328582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:34.329061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:34.329493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:34.330170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:34.330641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:34.332688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:34.459620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:34.467869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-506b7d63-830e-4c97-bdcc-e5887ce10560 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150104 Sep 18 06:58:34.468437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1559/2627] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:34 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:34.508997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:34.509602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:34.515438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:34.515918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:34.516350 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:34.517046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:34.517482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:34.517906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:34.642083 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:34.650154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-07862719-5d94-4c62-9511-0a122b0bee60 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.144821 Sep 18 06:58:34.651325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1560/2628] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:34 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:35.491875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:35.492491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:35.499003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:35.499494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:35.499937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:35.500625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:35.501108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:35.501783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:35.626505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:35.635074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-78d2103c-5da0-4b4c-80fb-47788ba572de tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147079 Sep 18 06:58:35.635645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1561/2629] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:35 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:35.673280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:35.673887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:35.679556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:35.680043 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:35.680482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:35.681165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:35.681621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:35.682078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:35.811574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:35.819704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c82b1a79-0b8e-42ad-ae8c-e8a19935f427 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149954 Sep 18 06:58:35.820280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1562/2630] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:35 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:36.658833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:36.659541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:36.665861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:36.666885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:36.667326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:36.668014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:36.668468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:36.668899 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:36.796197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:36.804432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b90b1536-9af4-4a34-9898-47a176a17739 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149584 Sep 18 06:58:36.804993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1563/2631] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:36 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:36.843146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:36.843764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:36.849252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:36.849725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:36.850155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:36.850850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:36.851298 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:36.851719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:36.982017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:36.990427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-eb3f5538-565b-4eef-a647-80bed4a17de7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150794 Sep 18 06:58:36.991026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1564/2632] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:36 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:37.828922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:37.829584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:37.835871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:37.836385 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:37.836824 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:37.837507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:37.838119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:37.838564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:37.964091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:37.972492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ac3d92af-b2bb-4b10-9a28-d3720079ab62 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147479 Sep 18 06:58:37.973370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1565/2633] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:37 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:38.012904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:38.013510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:38.019260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:38.019741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:38.020177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:38.020858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:38.021307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:38.021729 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:38.145028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:38.153950 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-17fcd4f1-9aed-4363-ba91-57f21afbb3a2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.144629 Sep 18 06:58:38.154649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1566/2634] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:38 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:38.996434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:38.997159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:39.003959 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:39.004440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:39.004876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:39.005556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:39.006011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:39.006436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:39.137980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:39.146178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ab42abb7-bc06-4611-8c2e-067803757d98 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.153829 Sep 18 06:58:39.146758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1567/2635] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:38 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:39.177205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:39.177807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:39.183857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:39.184341 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:39.184778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:39.185457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:39.185909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:39.186336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:39.309930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:39.318021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bed1ca1f-2da4-49e2-8855-ba56e57b4bfb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.144276 Sep 18 06:58:39.318613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1568/2636] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:39 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:40.169160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:40.169864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:40.176235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:40.176718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:40.177157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:40.177840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:40.178292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:40.178743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:40.307624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:40.315663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9b427dd5-2869-464a-87b5-ae3a1da1e6aa tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151082 Sep 18 06:58:40.316218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1569/2637] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:40 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:40.341103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:40.341697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:40.347172 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:40.347648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:40.348076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:40.348744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:40.349190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:40.349609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:40.483494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:40.486885 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:58:40.492554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cc11f036-736d-48fb-87eb-36e44d12aafe tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.155466 Sep 18 06:58:40.493112 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1570/2638] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:40 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:41.339677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:41.340368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:41.346859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:41.347338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:41.347769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:41.348448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:41.348910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:41.349312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:41.485604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:41.493781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4bf51298-fe27-4fce-8758-88deab1c6d1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.158483 Sep 18 06:58:41.494394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1571/2639] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:41 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:41.515869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:41.516467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:41.522081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:41.522621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:41.523032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:41.523707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:41.524153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:41.524572 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:41.650753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:41.661759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1c0576d7-d51e-41a8-99a6-8a0765209ceb tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149457 Sep 18 06:58:41.662325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1572/2640] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:41 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:42.517209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:42.517803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:42.524220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:42.524697 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:42.525130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:42.525803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:42.526252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:42.526690 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:42.651971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:42.660005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a76a0e83-bfb6-4766-9334-255f7e38d58b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146659 Sep 18 06:58:42.660566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1573/2641] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:42 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:42.687065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:42.687664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:42.693166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:42.693638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:42.694070 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:42.694777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:42.695229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:42.695649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:42.825937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:42.834321 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-77ce3b3a-03a3-4e64-89f4-04c0dc7e14a1 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150662 Sep 18 06:58:42.834925 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1574/2642] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:42 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:58:43.683652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:43.684314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:43.690672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:43.691198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:43.691586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:43.692258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:43.692733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:43.693150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:43.818344 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:43.826712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d4c66680-6642-42d3-98cf-73fa52fadb66 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.146894 Sep 18 06:58:43.827279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1575/2643] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:43 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:43.857302 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:43.857909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:43.863571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:43.864059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:43.864497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:43.865183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:43.865637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:43.866069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:43.994451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:44.002892 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3d1f7034-8eed-4c38-a762-37acd8b5d62e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149710 Sep 18 06:58:44.003458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1576/2644] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:43 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:44.850378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:44.851063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:44.857372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:44.857854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:44.858291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:44.859007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:44.859459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:44.859886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:44.986066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:44.994424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f5dce575-c834-434e-a0f2-79bec7a1bcbc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147909 Sep 18 06:58:44.995021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1577/2645] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:44 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:45.027280 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:45.028088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:45.034440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:45.034946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:45.035376 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:45.036062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:45.036709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:45.037427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:45.163998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:45.172152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-48ff215b-1aa7-4f1d-8b39-55e63f6db4cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149531 Sep 18 06:58:45.172713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1578/2646] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:45 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:46.018522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:46.019220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:46.026189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:46.026706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:46.027145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:46.027826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:46.028279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:46.028706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:46.159491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:46.168271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8a56e887-9f78-4847-9d12-efa3ebd58479 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.153934 Sep 18 06:58:46.168785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1579/2647] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:46 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:46.195017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:46.195616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:46.201531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:46.202006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:46.202438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:46.203213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:46.203657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:46.204072 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:46.331830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:46.340953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8050d83b-bc6d-4cf4-8a7b-e86672b9f4fc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149412 Sep 18 06:58:46.341554 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1580/2648] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:46 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:47.191514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:47.192176 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:47.198569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:47.199075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:47.199509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:47.200187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:47.200637 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:47.201058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:47.329331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:47.337888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-40cbbcf4-1053-4863-b434-a7cd22ac549e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149705 Sep 18 06:58:47.338445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1581/2649] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:47 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:47.363576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:47.364194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:47.371024 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:47.371512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:47.371954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:47.372640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:47.373095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:47.373522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:47.500348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:47.508863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-07c3754a-b798-4c09-93f4-6a2f7d9bb7cf tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148868 Sep 18 06:58:47.509427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1582/2650] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:47 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:48.361661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:48.362317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:48.365647 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:58:48.370031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:48.370997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:48.371523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:48.372715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:48.373184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:48.373615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:48.505425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:48.513588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7f30981a-8c17-435d-90b8-607d45dce043 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.155897 Sep 18 06:58:48.514145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1583/2651] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:48 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:48.531218 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:48.531859 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:48.537529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:48.538015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:48.538454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:48.539170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:48.539629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:48.540059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:48.667666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:48.675823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-097365d9-f16a-4bf8-bc2a-3255afba74e9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148255 Sep 18 06:58:48.676397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1584/2652] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:48 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:49.536751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:49.537364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:49.543653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:49.544143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:49.544626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:49.545256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:49.545710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:49.546135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:49.672566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:49.680671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9f96e53b-873a-41aa-8e65-ad23558a04ff tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147794 Sep 18 06:58:49.681727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1585/2653] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:49 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:49.699551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:49.700116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:49.705722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:49.706200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:49.706654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:49.707329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:49.707777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:49.708198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:49.835692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:49.843830 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4cf3ebc7-0999-4775-8ab0-c74b0bddd45f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148466 Sep 18 06:58:49.844364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1586/2654] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:49 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:58:50.703611 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:50.704285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:50.710634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:50.711115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:50.711552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:50.712968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:50.713460 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:50.713886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:50.731429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:58:50.840681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:50.848846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bae63722-6dfa-4a49-abf1-63d5dcb58ac8 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149120 Sep 18 06:58:50.849405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1587/2655] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:50 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:50.870143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:50.870827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:50.876849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:50.877334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:50.877770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:50.878452 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:50.878953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:50.879377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:51.008236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:51.016949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-934fe91a-d1d4-4fa0-8210-08794f501a25 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151728 Sep 18 06:58:51.017536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1588/2656] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:50 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:51.873174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:51.873808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:51.880417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:51.880895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:51.881326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:51.882003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:51.882449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:51.882904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:52.011868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:52.020328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0e98da96-767b-42cd-9675-6a6bb563c70c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150781 Sep 18 06:58:52.020891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1589/2657] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:51 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:52.040749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:52.041348 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:52.047062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:52.047537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:52.047967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:52.048641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:52.049089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:52.049509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:52.174186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:52.182909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bfa57866-0665-4b29-b758-1173f8407288 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.145715 Sep 18 06:58:52.183482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1590/2658] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:52 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:53.044865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:53.045531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:53.052082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:53.052561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:53.052992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:53.053676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:53.054127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:53.054545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:53.182853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:53.190905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d4934602-5183-4aa6-975e-a9366374e803 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152064 Sep 18 06:58:53.191468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1591/2659] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:53 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:53.204736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:53.205336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:53.210882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:53.211364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:53.211803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:53.212484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:53.212934 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:53.213356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:53.339159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:53.347272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e9a3c241-df15-438b-9d3f-83e82b9810e7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146115 Sep 18 06:58:53.347847 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1592/2660] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:53 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:54.214000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:54.214716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:54.221061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:54.221539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:54.221969 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:54.222669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:54.223118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:54.223539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:54.352555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:54.360610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9963ab0c-9033-48c9-ae44-507eeb856386 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150478 Sep 18 06:58:54.361169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1593/2661] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:54 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:54.371190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:54.371910 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:54.381936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:54.382430 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:54.382918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:54.383624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:54.384732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:54.385196 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:54.512325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:54.520587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ad0fb577-01d9-4342-821a-66b93bd64ac3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.153776 Sep 18 06:58:54.521154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1594/2662] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:54 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:55.384954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:55.385610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:55.392680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:55.393161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:55.393594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:55.394275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:55.394764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:55.395187 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:55.529911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:55.538190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3849408c-7458-4643-a388-691bec0cc74f tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.157586 Sep 18 06:58:55.538864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1595/2663] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:55 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:55.543250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:55.543836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:55.549393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:55.549877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:55.550316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:55.551025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:55.551479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:55.551904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:55.678624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:55.687277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34d08644-5d9c-4b2a-aafa-070de1b74509 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147467 Sep 18 06:58:55.687850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1596/2664] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:55 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:58:56.561192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:56.561810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:56.568277 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:56.568761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:56.569204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:56.569893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:56.570347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:56.570802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:56.697820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:56.706262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-917b1499-625d-46ed-86c4-dbebefcf8288 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.148933 Sep 18 06:58:56.706849 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1597/2665] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:56 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:56.711203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:56.711798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:56.717915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:56.718331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:56.718951 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:56.719638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:56.720077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:56.720495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:56.853362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:56.861571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-908f02d8-f356-4e57-861b-c8fc6b474c44 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.153802 Sep 18 06:58:56.862136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1598/2666] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:56 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:58:57.733443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:57.734092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:57.740587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:57.741066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:57.741499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:57.742173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:57.742641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:57.743658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:57.868220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:57.876688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-18190929-952d-479b-b55c-6b1807290c1a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147007 Sep 18 06:58:57.877260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1599/2667] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:57 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:57.884557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:57.885186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:57.890837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:57.891318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:57.891756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:57.892462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:57.892916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:57.893339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:58.021791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:58.030638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e72fa0f0-2289-4018-9b03-94ca19b46ece tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149717 Sep 18 06:58:58.031207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1600/2668] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:57 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:58.899827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:58.900502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:58.906810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:58.907290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:58.907722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:58.908398 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:58.908846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:58.909271 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:59.045961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:59.054894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fa96ed62-f327-42a6-8d9a-06cc4d2f1bca tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.159025 Sep 18 06:58:59.055492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1601/2669] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:58 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:58:59.060546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:58:59.061776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:58:59.066990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:59.067478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:59.067916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:59.068606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:58:59.069061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:58:59.069487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:58:59.199523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:58:59.207767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a2938831-7fe3-40bc-8d5a-f6760c01f00f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150750 Sep 18 06:58:59.208326 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1602/2670] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:58:59 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:00.078555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:00.079261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:00.085735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:00.086220 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:00.086688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:00.087373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:00.087827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:00.088255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:00.216549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:00.224743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fc6200ea-d9fb-440d-95a0-5a100f23de29 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150101 Sep 18 06:59:00.225338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1603/2671] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:00 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:00.230939 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:00.231540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:00.237078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:00.237553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:00.237986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:00.238689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:00.239140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:00.239562 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:00.243919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:00.369707 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:00.379144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-60f79552-31cf-488e-b265-6006e288d451 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151713 Sep 18 06:59:00.379721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1604/2672] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:00 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:01.248461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:01.249389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:01.258098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:01.259542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:01.260272 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:01.260986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:01.261447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:01.261877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:01.395087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:01.403902 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-941072bb-c2a1-455b-bce2-517dcff85372 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.159380 Sep 18 06:59:01.404472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1605/2673] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:01 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:01.408819 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:01.409415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:01.414970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:01.415472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:01.415911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:01.416595 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:01.417048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:01.417472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:01.544316 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:01.552559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-41820597-39ae-4a08-8b9b-bdbe1a9d9ce9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147157 Sep 18 06:59:01.553128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1606/2674] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:01 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:02.427035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:02.427717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:02.434126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:02.434633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:02.435077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:02.435758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:02.436215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:02.436640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:02.567533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:02.575630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0c849b89-5cbf-48de-a94b-03b73f2dca42 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152431 Sep 18 06:59:02.576198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1607/2675] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:02 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:02.636757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:02.637364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:02.643045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:02.643528 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:02.643963 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:02.644648 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:02.645100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:02.645525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:02.770804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:02.779600 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fa9a876e-8314-473e-bf5a-fb37a8a9c744 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146445 Sep 18 06:59:02.780173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1608/2676] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:02 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:03.599060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:03.599705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:03.606669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:03.607148 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:03.607582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:03.608257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:03.608704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:03.609119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:03.735927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:03.744857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9381f587-3852-434f-8f78-94e41cff8a76 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149689 Sep 18 06:59:03.745418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1609/2677] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:03 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:03.802802 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:03.803418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:03.809726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:03.810208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:03.810704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:03.811392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:03.811846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:03.812276 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:03.945018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:03.953243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a4afe7d3-1ce1-48a6-90b5-149069e2436b tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154086 Sep 18 06:59:03.953813 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1610/2678] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:03 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:04.768203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:04.768836 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:04.775255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:04.775744 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:04.776185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:04.776875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:04.777334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:04.777762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:04.903995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:04.912144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-908a9d0e-786b-45d3-93df-3f0aa6a044e5 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.147890 Sep 18 06:59:04.912706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1611/2679] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:04 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:04.976215 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:04.976815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:04.983003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:04.983471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:04.983897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:04.984566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:04.985010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:04.985425 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:05.118379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:05.126918 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bd21fb14-3f99-4beb-99fe-ed4070848b4e tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154254 Sep 18 06:59:05.127487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1612/2680] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:04 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:05.935494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:05.936156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:05.942482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:05.943005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:05.943441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:05.944146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:05.944593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:05.945010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:06.073558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:06.081898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d27d6ac8-571a-4b19-a45e-2fadb26b7278 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150250 Sep 18 06:59:06.082480 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1613/2681] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:05 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:06.149717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:06.150322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:06.155979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:06.156462 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:06.156901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:06.157586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:06.158046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:06.158471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:06.283322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:06.291933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5b5aed5a-cc34-49cd-afc1-763f7d7791dc tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146251 Sep 18 06:59:06.292514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1614/2682] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:06 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:07.106092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:07.106791 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:07.113130 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:07.113635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:07.114054 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:07.114775 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:07.115233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:07.115661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:07.247231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:07.255785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dfdb99ba-850f-444b-bbb3-d451c2833b7b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154164 Sep 18 06:59:07.256352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1615/2683] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:07 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:07.314567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:07.315198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:07.321548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:07.322059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:07.322494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:07.323197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:07.323652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:07.324075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:07.449392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:07.458137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-69cb37e2-2bde-40ef-bf68-11e1d58f1170 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.147181 Sep 18 06:59:07.465598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1616/2684] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:07 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:08.279360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:08.280046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:08.286364 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:08.286873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:08.287312 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:08.287993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:08.288448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:08.288873 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:08.418772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:08.426827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dd0deb8e-8d03-4033-bdf6-94bf503a2802 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151306 Sep 18 06:59:08.427379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1617/2685] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:08 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:08.481467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:08.482071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:08.488077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:08.488552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:08.488982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:08.489654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:08.490097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:08.490514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:08.615735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:08.623823 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7bd74d85-db1a-4668-a768-0cbba038a4f7 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146621 Sep 18 06:59:08.624387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1618/2686] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:08 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:09.449733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:09.450355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:09.457100 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:09.457589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:09.458028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:09.458765 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:09.459226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:09.459652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:09.591590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:09.599869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-33710897-3027-4491-9fa5-e89c63963cb9 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.154081 Sep 18 06:59:09.600436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1619/2687] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:09 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:09.647214 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:09.647820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:09.654018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:09.654534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:09.654999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:09.655693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:09.656147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:09.656574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:09.784917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:09.788393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:09.794509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-53576c07-f146-4f58-93e7-3bcc5b4e5d9f tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151085 Sep 18 06:59:09.795105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1620/2688] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:09 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:09.851596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:09.852738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:09.858672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:09.859153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:09.859587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:09.860264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:09.860715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:09.861132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:09.885478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:59:09.960294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-35d4398a-09e7-4389-952c-6b50a849c551 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 204 len: 0 microversion: 2.1 time: 0.163849 Sep 18 06:59:09.961032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1621/2689] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:59:09 2022] DELETE /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 0 bytes in 165 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:59:09.972778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:09.973850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:09.982880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:09.983584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:09.984232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:09.985259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:09.985933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:09.986569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:10.121639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:10.129583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b10a95f5-6e8b-4a7f-b57f-4f01e90e35e1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1681 microversion: 2.1 time: 0.163207 Sep 18 06:59:10.130150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1622/2690] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:09 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1681 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:10.624789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:10.625444 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:10.632088 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:10.632563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:10.633575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:10.634307 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:10.634786 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:10.635207 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:10.764785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:10.772905 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-47af7e20-aa31-4eeb-a292-5812d380fefe tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.152065 Sep 18 06:59:10.773465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1623/2691] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:10 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:10.817962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:10.818563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:10.824717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:10.825179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:10.825608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:10.826285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:10.826772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:10.827216 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:10.952921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:10.961432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-61086e60-2a07-4812-8613-22bb602ff999 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.146969 Sep 18 06:59:10.962151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1624/2692] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:10 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:11.153243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:11.153861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:11.159844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:11.160331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:11.160778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:11.161469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:11.161923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:11.162352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:11.331881 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:11.340113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dfa77804-ba6d-4e77-b1c5-be22d7c607dc tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1681 microversion: 2.1 time: 0.190720 Sep 18 06:59:11.340689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1625/2693] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:11 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1681 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:11.796079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:11.796725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:11.803178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:11.803664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:11.804104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:11.804795 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:11.805368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:11.805804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:12.019499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:12.028401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3dd256cf-4779-4bdf-8636-5322e29a073e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.236331 Sep 18 06:59:12.042598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1626/2694] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:11 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:12.042598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:12.042598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:12.042598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:12.043077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:12.043077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:12.043077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:12.043077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:12.043468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:12.256855 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:12.270609 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-06108d59-f457-432d-814c-c921b2541d5d tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.240124 Sep 18 06:59:12.271379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1627/2695] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:12 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:12.276005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-5ada31cf-9906-4101-a47a-65dd4253f58c req-ee284f54-654f-45e8-bd25-068c9254928a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48431e56-3b9d-4dbe-b0ad-10c0c59f12ae", "name": "network-vif-deleted", "tag": "5fbe700a-3ce0-47a3-b833-ae3c3099e861"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:59:12.289226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5ada31cf-9906-4101-a47a-65dd4253f58c req-ee284f54-654f-45e8-bd25-068c9254928a service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:12.289709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5ada31cf-9906-4101-a47a-65dd4253f58c req-ee284f54-654f-45e8-bd25-068c9254928a service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:12.290151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5ada31cf-9906-4101-a47a-65dd4253f58c req-ee284f54-654f-45e8-bd25-068c9254928a service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:12.303646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-5ada31cf-9906-4101-a47a-65dd4253f58c req-ee284f54-654f-45e8-bd25-068c9254928a service nova] Creating event network-vif-deleted:5fbe700a-3ce0-47a3-b833-ae3c3099e861 for instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae on hv-cinder-85535 Sep 18 06:59:12.309369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-5ada31cf-9906-4101-a47a-65dd4253f58c req-ee284f54-654f-45e8-bd25-068c9254928a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036866 Sep 18 06:59:12.311105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1628/2696] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:59:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:59:12.365414 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:12.366105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:12.373198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:12.373693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:12.374180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:12.374924 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:12.375413 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:12.375838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:12.408375 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:12.410483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:12.410994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:12.411536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:12.412197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lazy-loading 'fault' on Instance uuid 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:59:12.461063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd85160b01d824bc3a32b425fab98d2c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c5d2a3f2d944529b8c505fa23b5a10f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:12.470841 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9f9add69-08f0-427b-87cb-90c365179a91 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 200 len: 1329 microversion: 2.1 time: 0.110775 Sep 18 06:59:12.471410 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1629/2697] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:12 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 1329 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:13.053245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:13.053958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:13.060533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:13.061022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:13.061458 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:13.062160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:13.062636 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:13.063066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:13.200113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:13.208521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1a96cd1c-325f-4898-a374-a8d747c9eb16 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.159378 Sep 18 06:59:13.209161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1630/2698] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:13 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:13.295230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:13.295965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:13.301996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:13.302474 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:13.302938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:13.303619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:13.304064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:13.304483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:13.430999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:13.439188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9093fe3c-4af4-4cd1-a023-a55e1cf110e0 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148712 Sep 18 06:59:13.439756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1631/2699] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:13 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:13.495030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:13.495652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:13.501804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:13.502285 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:13.502884 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:13.503571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:13.504022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:13.504448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:13.517670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae HTTP/1.1 Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:13.519366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:13.521206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '48431e56-3b9d-4dbe-b0ad-10c0c59f12ae'} Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:13.522241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:13.524006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:13.525779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:13.525779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-baf0f0a2-e222-4f7c-8839-4dd4f35fd0dd tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 404 len: 111 microversion: 2.1 time: 0.030346 Sep 18 06:59:13.525779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1632/2700] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:13 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 06:59:13.528974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0a228406-be36-40ad-b79e-6571e1b4e7d1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '.version_select at 0x7fe67052e940>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:13.547448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0a228406-be36-40ad-b79e-6571e1b4e7d1 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-66806330" status: 202 len: 0 microversion: 2.1 time: 0.021870 Sep 18 06:59:13.548018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1633/2701] 10.222.0.20 () {58 vars in 1215 bytes} [Sun Sep 18 06:59:13 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-66806330 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 06:59:14.172237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:14.172877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:14.179476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.180578 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.181015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.181688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.182142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.182564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.195359 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae HTTP/1.1 Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:14.196909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.198710 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '48431e56-3b9d-4dbe-b0ad-10c0c59f12ae'} Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:14.199613 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.201049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.202673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:14.202673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-13a424dc-7e25-4045-9f5a-fb6f00322139 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 404 len: 111 microversion: 2.1 time: 0.030638 Sep 18 06:59:14.202673 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1634/2702] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:59:14 2022] DELETE /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 06:59:14.205889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:14.206488 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:14.212103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.212582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.213018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.213695 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.214145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.214567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.227908 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae HTTP/1.1 Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:14.229442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.231202 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '48431e56-3b9d-4dbe-b0ad-10c0c59f12ae'} Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:14.232177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. Sep 18 06:59:14.233779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.235208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance 48431e56-3b9d-4dbe-b0ad-10c0c59f12ae could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:14.235208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e7f9fbb-b5f8-48b2-8348-ebd08e729d1a tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae" status: 404 len: 111 microversion: 2.1 time: 0.028339 Sep 18 06:59:14.235208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1635/2703] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:14 2022] GET /compute/v2.1/servers/48431e56-3b9d-4dbe-b0ad-10c0c59f12ae => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:59:14.237559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:14.238155 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:14.244357 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.244829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.245262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.245936 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.246386 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.246844 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.381720 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:14.390323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-e2825c4a-fca6-41bc-84b5-cbf6aaa2d282 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.157848 Sep 18 06:59:14.390913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1636/2704] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:14 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:14.395644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:14.396149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:14.402071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.402549 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.403015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.403692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.404141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.404565 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.419290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b HTTP/1.1 Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:14.420817 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.422522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'ceaad6c7-d3bd-45ed-9e68-010c68b41f1b'} Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:14.423418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.424852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.426447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:14.426447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3e870f4f-401c-40e6-ac76-8dcd1439fd2c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 404 len: 111 microversion: 2.1 time: 0.030343 Sep 18 06:59:14.426447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1637/2705] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:59:14 2022] DELETE /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:59:14.431693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:14.432291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: ceaad6c7-d3bd-45ed-9e68-010c68b41f1b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:14.438257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.438807 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.439247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.439973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.440466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.440876 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.454514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b HTTP/1.1 Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:14.456090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.457826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'ceaad6c7-d3bd-45ed-9e68-010c68b41f1b'} Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:14.458766 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. Sep 18 06:59:14.460250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.462115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance ceaad6c7-d3bd-45ed-9e68-010c68b41f1b could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:14.462115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b2e6dd41-0551-42fe-83ea-3f764b57cf3c tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b" status: 404 len: 111 microversion: 2.1 time: 0.029603 Sep 18 06:59:14.462115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1638/2706] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:14 2022] GET /compute/v2.1/servers/ceaad6c7-d3bd-45ed-9e68-010c68b41f1b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:59:14.463013 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:14.463619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:14.469475 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.469956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.470389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.471108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.471559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.471984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.601449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:14.610201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9a6a4b59-eace-4fc7-8eb8-759ee96588d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.151292 Sep 18 06:59:14.610798 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1639/2707] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:14 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:14.615175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:14.615671 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:14.621259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.621743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.622183 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.622895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.623352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.623774 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.637090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd HTTP/1.1 Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:14.638688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.640428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '2131eb93-ac09-4fc4-991e-3a0cc04b5bbd'} Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:14.641403 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.643048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.644752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:14.644752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0df8ca0d-e7e6-48a6-8dca-638bdd8d1cb5 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 404 len: 111 microversion: 2.1 time: 0.028342 Sep 18 06:59:14.644752 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1640/2708] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:59:14 2022] DELETE /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:59:14.648028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:14.649226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:14.656060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.656542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.656979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.657664 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:14.658118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:14.658544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:14.662717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:14.674663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd HTTP/1.1 Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:14.676491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.678240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '2131eb93-ac09-4fc4-991e-3a0cc04b5bbd'} Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:14.679286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. Sep 18 06:59:14.680973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:14.682715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance 2131eb93-ac09-4fc4-991e-3a0cc04b5bbd could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:14.682715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2713f74c-8731-4a7f-9d9b-ad77983c45d8 tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd" status: 404 len: 111 microversion: 2.1 time: 0.033347 Sep 18 06:59:14.682715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1641/2709] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:14 2022] GET /compute/v2.1/servers/2131eb93-ac09-4fc4-991e-3a0cc04b5bbd => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 06:59:15.413821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:15.415068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:15.422853 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:15.423333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:15.423768 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:15.424450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:15.424904 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:15.425331 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:15.552322 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:15.561084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-56a7c8b9-503c-4d00-865b-043cc03e6114 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151098 Sep 18 06:59:15.561641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1642/2710] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:15 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:15.632831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:15.633429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:15.639184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:15.639660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:15.640094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:15.640764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:15.641210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:15.641628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:15.769122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:15.777495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9d1e1ca1-b828-45f6-b9fa-ce1408904a1c tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.148264 Sep 18 06:59:15.778060 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1643/2711] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:15 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:16.584469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:16.585120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:16.591561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:16.592038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:16.592472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:16.593150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:16.593594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:16.594014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:16.734309 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:16.743002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d1194dca-da32-4a09-8cdf-4c1fd84c106c tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.162667 Sep 18 06:59:16.743556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1644/2712] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:16 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:16.800315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:16.800912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:16.806990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:16.807464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:16.807893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:16.808563 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:16.809037 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:16.809457 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:16.942716 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:16.951560 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6f6c2e73-24a9-41dd-8656-d16c23068322 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.154692 Sep 18 06:59:16.952135 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1645/2713] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:16 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:17.766790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:17.767443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:17.773627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:17.774106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:17.774541 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:17.775240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:17.775685 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:17.776102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:17.905945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:17.914497 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-20e90de9-7699-4698-8b59-4127635dca31 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.151118 Sep 18 06:59:17.915159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1646/2714] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:17 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:17.973947 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:17.974546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:17.980730 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:17.981204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:17.981633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:17.982317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:17.982806 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:17.983224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:18.111979 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:18.120252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-654060e8-1019-448d-b5c2-a17bec571934 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149854 Sep 18 06:59:18.120821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1647/2715] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:17 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:18.937262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:18.937932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:18.944005 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:18.944481 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:18.944942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:18.946338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:18.946846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:18.947284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:19.074870 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:19.083076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-20a71216-709b-4aa2-affb-2c153dac3809 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.149655 Sep 18 06:59:19.083649 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1648/2716] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:18 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:19.143539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:19.144149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:19.149879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:19.150383 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:19.151199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:19.152319 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:19.153025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:19.153827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:19.280504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:19.290098 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ed80481f-89ea-485f-a9d3-7cfd1323eeb4 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.149879 Sep 18 06:59:19.290993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1649/2717] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:19 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:20.106792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:20.107355 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:20.113846 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:20.114324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:20.114808 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:20.115485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:20.115930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:20.116349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:20.244735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:20.253177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f4a305f9-92a5-4612-9241-92c533e8ddb6 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1734 microversion: 2.1 time: 0.150371 Sep 18 06:59:20.253741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1650/2718] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:20 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:20.314931 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:20.315548 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:20.322010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:20.322489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:20.322952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:20.323638 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:20.324093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:20.324521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:20.453114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:20.461626 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34e62706-893c-408c-a468-1f2c09436b90 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.150271 Sep 18 06:59:20.462253 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1651/2719] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:20 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:20.932428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-51f4d4bc-7538-4bf8-943b-381d3c92bee3) Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 HTTP/1.1 Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:59:20.935748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:59:20.937405 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-51f4d4bc-7538-4bf8-943b-381d3c92bee3) Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'server_id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7', 'id': '0dd5997f-7321-4086-9598-6cca845c9907'} Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:59:20.938129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-51f4d4bc-7538-4bf8-943b-381d3c92bee3) Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 06:59:20.939528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-51f4d4bc-7538-4bf8-943b-381d3c92bee3) Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:20.940874 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-51f4d4bc-7538-4bf8-943b-381d3c92bee3) Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:20.942184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-51f4d4bc-7538-4bf8-943b-381d3c92bee3) Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-51f4d4bc-7538-4bf8-943b-381d3c92bee3) Sep 18 06:59:20.945069 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:20.948436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-51f4d4bc-7538-4bf8-943b-381d3c92bee3) {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:20.948436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907" status: 400 len: 269 microversion: 2.1 time: 235.265008 Sep 18 06:59:20.948436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:59:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (ip 10.222.0.20) !!! Sep 18 06:59:20.948436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:59:20 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 (10.222.0.20) Sep 18 06:59:20.948436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: CRITICAL nova [None req-6930c708-d465-4bac-85d0-9838e24141d9 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Unhandled error: OSError: write error Sep 18 06:59:20.948436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova OSError: write error Sep 18 06:59:20.948436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova Sep 18 06:59:20.949345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1069/2720] 10.222.0.20 () {58 vars in 1416 bytes} [Sun Sep 18 06:55:25 2022] DELETE /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7/os-volume_attachments/0dd5997f-7321-4086-9598-6cca845c9907 => generated 0 bytes in 235267 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 18 06:59:21.283619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:21.284221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:21.293898 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:21.294393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:21.294856 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:21.295906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:21.296360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:21.297052 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:21.449119 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:21.488533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:21.489426 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:21.491394 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c871ccc6-58c6-4c92-95eb-6af344c482d0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1688 microversion: 2.1 time: 0.206679 Sep 18 06:59:21.495472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1652/2721] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:21 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1688 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:21.499270 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:21.499765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:21.500198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:21.501162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:21.501611 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:21.502410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:21.675294 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:21.685304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-09c251a6-7323-4069-ab38-00147f7a9b20 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1561 microversion: 2.1 time: 0.204180 Sep 18 06:59:21.685875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1070/2722] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:21 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1561 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:22.519028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:22.520513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:22.532470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:22.534270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:22.535149 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:22.536448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:22.537594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:22.538200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:22.711772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:22.713046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:22.721060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:22.721823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:22.722258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:22.723201 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:22.723824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:22.724417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:22.800254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:22.812862 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b653e849-0de7-4e7d-bb14-1feb0892b356 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1620 microversion: 2.1 time: 0.297249 Sep 18 06:59:22.813725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1653/2723] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:22 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1620 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:22.986736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:22.996410 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-91ef84cd-ec7d-4751-9546-9a6103b290e2 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1515 microversion: 2.1 time: 0.290576 Sep 18 06:59:22.997571 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1071/2724] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:22 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1515 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:23.100434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-0b615c49-0170-4ffb-813a-cff28394eed8 req-bbc1960a-5f7f-4400-abf1-7043a6d2e773 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a8254cee-e629-4d82-988c-708bac734c1e", "name": "network-vif-deleted", "tag": "7ff7d3c9-95be-4eea-b0b3-a8bd5d358217"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:59:23.121153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-0b615c49-0170-4ffb-813a-cff28394eed8 req-bbc1960a-5f7f-4400-abf1-7043a6d2e773 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:23.123256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-0b615c49-0170-4ffb-813a-cff28394eed8 req-bbc1960a-5f7f-4400-abf1-7043a6d2e773 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:23.123751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-0b615c49-0170-4ffb-813a-cff28394eed8 req-bbc1960a-5f7f-4400-abf1-7043a6d2e773 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:23.158865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-0b615c49-0170-4ffb-813a-cff28394eed8 req-bbc1960a-5f7f-4400-abf1-7043a6d2e773 service nova] Creating event network-vif-deleted:7ff7d3c9-95be-4eea-b0b3-a8bd5d358217 for instance a8254cee-e629-4d82-988c-708bac734c1e on hv-cinder-85535 Sep 18 06:59:23.169011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-0b615c49-0170-4ffb-813a-cff28394eed8 req-bbc1960a-5f7f-4400-abf1-7043a6d2e773 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073298 Sep 18 06:59:23.170249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1654/2725] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:59:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:59:23.668185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-cb9e9792-cc3d-4d7d-bc13-c0324ab1b2a0 req-c51b2be6-dbbd-4df7-8f3f-c9af3b9f4b6f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7", "name": "network-vif-deleted", "tag": "3b32ba19-f05b-4744-adab-ddcb74d679c1"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:59:23.681993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-cb9e9792-cc3d-4d7d-bc13-c0324ab1b2a0 req-c51b2be6-dbbd-4df7-8f3f-c9af3b9f4b6f service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:23.682458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-cb9e9792-cc3d-4d7d-bc13-c0324ab1b2a0 req-c51b2be6-dbbd-4df7-8f3f-c9af3b9f4b6f service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:23.688505 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-cb9e9792-cc3d-4d7d-bc13-c0324ab1b2a0 req-c51b2be6-dbbd-4df7-8f3f-c9af3b9f4b6f service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:23.701845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-cb9e9792-cc3d-4d7d-bc13-c0324ab1b2a0 req-c51b2be6-dbbd-4df7-8f3f-c9af3b9f4b6f service nova] Creating event network-vif-deleted:3b32ba19-f05b-4744-adab-ddcb74d679c1 for instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 on hv-cinder-85535 Sep 18 06:59:23.711738 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-cb9e9792-cc3d-4d7d-bc13-c0324ab1b2a0 req-c51b2be6-dbbd-4df7-8f3f-c9af3b9f4b6f service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.476804 Sep 18 06:59:23.712507 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1072/2726] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:59:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:59:23.847732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:23.849127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:23.862738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:23.863623 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:23.864090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:23.864857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:23.865315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:23.866016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:23.918735 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:23.919227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:23.919851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:23.920670 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lazy-loading 'fault' on Instance uuid a8254cee-e629-4d82-988c-708bac734c1e {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:59:23.970025 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:23.986093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f36fb386-0ef4-480c-8368-e6b874088371 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 200 len: 1335 microversion: 2.1 time: 0.152430 Sep 18 06:59:23.987349 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1655/2727] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:23 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 1335 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:24.024327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:24.038328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:24.053658 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:24.054602 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:24.056287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:24.057767 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:24.060386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:24.061519 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:24.101075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:24.102755 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:24.103278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:24.103744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lazy-loading 'fault' on Instance uuid e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:59:24.152836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e702f6ad9e38492497e0ae33abb9aae7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d431bdc9a2244dcb1bb589dee792d02', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:24.162884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3445e30e-64f2-41e9-abaf-29e691d26faa tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 200 len: 1309 microversion: 2.1 time: 0.143484 Sep 18 06:59:24.166718 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1073/2728] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:24 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 1309 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:25.010205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:25.010912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: a8254cee-e629-4d82-988c-708bac734c1e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:25.017466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:25.017946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:25.018379 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:25.019086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:25.019534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:25.019954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:25.033954 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] HTTP exception thrown: Instance a8254cee-e629-4d82-988c-708bac734c1e could not be found. Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Request method failure captured: Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e HTTP/1.1 Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmU55Wg3gBiYn2aJSEvLyVJMV1h4CCP-z1W3AffyLY3fsW3JQ8OjuZv4iFhhTSFmKumF1s37EILY-hv50aLHoH2J1-083C3bmZIbw6-XE242ooMA_PlrxaMx2_DE94obNE1dW82mTeLq0C4XT8RYmdFLgmo0_0kmRhbQKjKMrBjrZ0 Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:25.035620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_91ef3478fd574180abe99393ba769124"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/91ef3478fd574180abe99393ba769124"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 7a6b8fbd31464e4d99449c74b4d1dda5 Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance a8254cee-e629-4d82-988c-708bac734c1e could not be found. Sep 18 06:59:25.037402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'a8254cee-e629-4d82-988c-708bac734c1e'} Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a8254cee-e629-4d82-988c-708bac734c1e could not be found. Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:25.038378 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a8254cee-e629-4d82-988c-708bac734c1e could not be found. Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a8254cee-e629-4d82-988c-708bac734c1e could not be found. Sep 18 06:59:25.040818 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.042141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Returning 404 to user: Instance a8254cee-e629-4d82-988c-708bac734c1e could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:25.042141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2558491e-a76f-4dbd-a92e-ce5310d80b7e tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e" status: 404 len: 111 microversion: 2.1 time: 0.030845 Sep 18 06:59:25.042141 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1656/2729] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:25 2022] GET /compute/v2.1/servers/a8254cee-e629-4d82-988c-708bac734c1e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:59:25.046057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-86086525-9b9d-43c7-937b-c8af57f109bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '.version_select at 0x7fe67066f670>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:25.082709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-86086525-9b9d-43c7-937b-c8af57f109bb tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-164943999" status: 202 len: 0 microversion: 2.1 time: 0.040229 Sep 18 06:59:25.083306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1074/2730] 10.222.0.20 () {58 vars in 1218 bytes} [Sun Sep 18 06:59:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-164943999 => generated 0 bytes in 42 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 06:59:25.193754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:25.194484 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] [instance: e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:25.201438 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:25.202021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:25.202529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:25.203588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:25.204232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:25.204568 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:25.232114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] HTTP exception thrown: Instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 could not be found. Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Request method failure captured: Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 HTTP/1.1 Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrodXASZoc2QfcLsDSI4tv5ay3Wcd8H8L9BhuodlC3XSyRVg34EaXnUE4_FF_l3h5lYqNh9l0KK2mMVbmmfDNAV2B1iUv9b7YPmsRk1aLqqanTCGzTrAe8rmvltZ6TBOrYYdXXUs2xJGItmA3gC8FIfWanE0dqpkor-MfDo4lo-bT0Q Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:25.233767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/1d431bdc9a2244dcb1bb589dee792d02"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 1d431bdc9a2244dcb1bb589dee792d02 Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1272927719 Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: e702f6ad9e38492497e0ae33abb9aae7 Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesSnapshotTestJSON-1272927719-project Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 could not be found. Sep 18 06:59:25.235914 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7'} Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 could not be found. Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:25.236919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 could not be found. Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 could not be found. Sep 18 06:59:25.238512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:25.240324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] Returning 404 to user: Instance e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:25.240324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8b520559-e052-45cd-95b8-7e009b2763d3 tempest-VolumesSnapshotTestJSON-1272927719 tempest-VolumesSnapshotTestJSON-1272927719-project] 10.222.0.20 "GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7" status: 404 len: 111 microversion: 2.1 time: 0.051029 Sep 18 06:59:25.240324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1657/2731] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:25 2022] GET /compute/v2.1/servers/e84fb7f8-37c6-419f-9ff6-0f6bfc4024c7 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:59:33.366725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 06:59:33.367250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 06:59:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 (ip 10.222.0.20) !!! Sep 18 06:59:33.373073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:33.373467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:33.381174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:33.381682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:33.382115 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:33.382841 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:33.383300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:33.383734 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:33.411736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Going to try to terminate instance {{(pid=120380) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:59:33.497565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0bd9c28a-565e-4b03-9897-adf5a497943b tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 204 len: 0 microversion: 2.1 time: 0.131513 Sep 18 06:59:33.498189 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1075/2732] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:59:33 2022] DELETE /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:59:33.507288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:33.507897 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:33.515507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:33.516007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:33.516431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:33.517174 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:33.517680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:33.518136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:33.647205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:33.655601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c79631ea-9fee-4f9a-b416-469bd110f46a tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1507 microversion: 2.1 time: 0.152432 Sep 18 06:59:33.656136 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1658/2733] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:33 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1507 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:34.513489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-e442dce8-87ab-463a-97d0-2ae0455d7622 req-98345b6a-2e17-4b7d-912b-71b759153af8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d8351557-8730-42b4-9788-684192b14e98", "name": "network-vif-deleted", "tag": "b4df9936-75d5-4973-a602-6e13994ffd2a"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:59:34.528613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-e442dce8-87ab-463a-97d0-2ae0455d7622 req-98345b6a-2e17-4b7d-912b-71b759153af8 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:34.529413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-e442dce8-87ab-463a-97d0-2ae0455d7622 req-98345b6a-2e17-4b7d-912b-71b759153af8 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:34.530062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-e442dce8-87ab-463a-97d0-2ae0455d7622 req-98345b6a-2e17-4b7d-912b-71b759153af8 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:34.544331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-e442dce8-87ab-463a-97d0-2ae0455d7622 req-98345b6a-2e17-4b7d-912b-71b759153af8 service nova] Creating event network-vif-deleted:b4df9936-75d5-4973-a602-6e13994ffd2a for instance d8351557-8730-42b4-9788-684192b14e98 on hv-cinder-85535 Sep 18 06:59:34.550375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-e442dce8-87ab-463a-97d0-2ae0455d7622 req-98345b6a-2e17-4b7d-912b-71b759153af8 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041118 Sep 18 06:59:34.551203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1076/2734] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:59:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:59:34.680655 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:34.682401 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:34.690116 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:34.690632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:34.691074 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:34.691771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:34.692226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:34.693893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:34.726949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:34.727672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:34.728325 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:34.728983 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lazy-loading 'fault' on Instance uuid d8351557-8730-42b4-9788-684192b14e98 {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:59:34.774503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:34.778026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=100 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:34.784097 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5d82ca0f-a325-4c7e-92db-b617537a45dc tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 200 len: 1312 microversion: 2.1 time: 0.107230 Sep 18 06:59:34.794625 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1659/2735] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:34 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 1312 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:35.808514 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:35.809210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: d8351557-8730-42b4-9788-684192b14e98] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:35.815622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:35.816098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:35.816532 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:35.817212 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:35.817660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:35.818080 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:35.832973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] HTTP exception thrown: Instance d8351557-8730-42b4-9788-684192b14e98 could not be found. Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Request method failure captured: Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 HTTP/1.1 Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmU55Wg3gBiYn2aJSEvLyVJMV1h4CCP-z1W3AffyLY3fsW3JQ8OjuZv4iFhhTSFmKumF1s37EILY-hv50aLHoH2J1-083C3bmZIbw6-XE242ooMA_PlrxaMx2_DE94obNE1dW82mTeLq0C4XT8RYmdFLgmo0_0kmRhbQKjKMrBjrZ0 Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 06:59:35.834668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_91ef3478fd574180abe99393ba769124"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/91ef3478fd574180abe99393ba769124"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 7a6b8fbd31464e4d99449c74b4d1dda5 Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance d8351557-8730-42b4-9788-684192b14e98 could not be found. Sep 18 06:59:35.836509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': 'd8351557-8730-42b4-9788-684192b14e98'} Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d8351557-8730-42b4-9788-684192b14e98 could not be found. Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:35.838735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d8351557-8730-42b4-9788-684192b14e98 could not be found. Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d8351557-8730-42b4-9788-684192b14e98 could not be found. Sep 18 06:59:35.840589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 06:59:35.842365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Returning 404 to user: Instance d8351557-8730-42b4-9788-684192b14e98 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:35.842365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4958c8aa-b713-4773-b9c2-0dda74a5037d tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98" status: 404 len: 111 microversion: 2.1 time: 0.031759 Sep 18 06:59:35.842365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1077/2736] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:35 2022] GET /compute/v2.1/servers/d8351557-8730-42b4-9788-684192b14e98 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 06:59:35.845199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:35.845688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:35.851991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:35.852463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:35.852973 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:35.853650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:35.854096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:35.854517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:35.883942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 06:59:35.969689 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ea9bacba-825b-4962-8c10-b719c9180037 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 204 len: 0 microversion: 2.1 time: 0.127966 Sep 18 06:59:35.972552 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1660/2737] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 06:59:35 2022] DELETE /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 06:59:35.980978 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:35.981948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:35.988420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:35.988939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:35.989418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:35.990269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:35.990793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:35.991447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:36.147757 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:36.155469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7bad504d-798f-4678-ad27-7590cb660cf0 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1522 microversion: 2.1 time: 0.178695 Sep 18 06:59:36.156040 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1078/2738] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:35 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 06:59:36.953619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-51b0af26-3f49-4698-858f-adcd12e11fae req-f2e4bb13-1d7f-4b37-9b6a-16d22bf4c4a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1dd95cf5-d6be-4de9-9941-8b1d35855bcb", "name": "network-vif-deleted", "tag": "76758f98-a1a3-4d17-a73a-67726f56e2d5"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 06:59:36.965644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-51b0af26-3f49-4698-858f-adcd12e11fae req-f2e4bb13-1d7f-4b37-9b6a-16d22bf4c4a2 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:36.966559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-51b0af26-3f49-4698-858f-adcd12e11fae req-f2e4bb13-1d7f-4b37-9b6a-16d22bf4c4a2 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:36.967255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-51b0af26-3f49-4698-858f-adcd12e11fae req-f2e4bb13-1d7f-4b37-9b6a-16d22bf4c4a2 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:36.979569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-51b0af26-3f49-4698-858f-adcd12e11fae req-f2e4bb13-1d7f-4b37-9b6a-16d22bf4c4a2 service nova] Creating event network-vif-deleted:76758f98-a1a3-4d17-a73a-67726f56e2d5 for instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb on hv-cinder-85535 Sep 18 06:59:36.984646 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-51b0af26-3f49-4698-858f-adcd12e11fae req-f2e4bb13-1d7f-4b37-9b6a-16d22bf4c4a2 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035075 Sep 18 06:59:36.985242 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1661/2739] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 06:59:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 06:59:37.179392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:37.180098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:37.186981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:37.187586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:37.188127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:37.188923 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:37.189455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:37.189976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:37.231243 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:37.232370 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:37.233064 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:37.233782 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lazy-loading 'fault' on Instance uuid 1dd95cf5-d6be-4de9-9941-8b1d35855bcb {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 06:59:37.288009 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7a6b8fbd31464e4d99449c74b4d1dda5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91ef3478fd574180abe99393ba769124', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 06:59:37.295818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fe154a3f-ca2e-46e8-b1ee-7fd65b824494 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 200 len: 1312 microversion: 2.1 time: 0.120243 Sep 18 06:59:37.296380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1079/2740] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:37 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 1312 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 06:59:38.323463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 06:59:38.324169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] [instance: 1dd95cf5-d6be-4de9-9941-8b1d35855bcb] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 06:59:38.330393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:38.330922 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:38.331369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:38.332080 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 06:59:38.332538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 06:59:38.332966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 06:59:38.348264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] HTTP exception thrown: Instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb could not be found. Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Request method failure captured: Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb HTTP/1.1 Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmU55Wg3gBiYn2aJSEvLyVJMV1h4CCP-z1W3AffyLY3fsW3JQ8OjuZv4iFhhTSFmKumF1s37EILY-hv50aLHoH2J1-083C3bmZIbw6-XE242ooMA_PlrxaMx2_DE94obNE1dW82mTeLq0C4XT8RYmdFLgmo0_0kmRhbQKjKMrBjrZ0 Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 06:59:38.349920 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_91ef3478fd574180abe99393ba769124"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/91ef3478fd574180abe99393ba769124"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/91ef3478fd574180abe99393ba769124"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 91ef3478fd574180abe99393ba769124 Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1922759309 Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 7a6b8fbd31464e4d99449c74b4d1dda5 Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1922759309-project Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb could not be found. Sep 18 06:59:38.351734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '1dd95cf5-d6be-4de9-9941-8b1d35855bcb'} Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb could not be found. Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 06:59:38.352746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb could not be found. Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb could not be found. Sep 18 06:59:38.355012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 06:59:38.356466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] Returning 404 to user: Instance 1dd95cf5-d6be-4de9-9941-8b1d35855bcb could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 06:59:38.356466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-235dcf76-b9e4-49fd-9fb4-8886be242988 tempest-ServerRescueNegativeTestJSON-1922759309 tempest-ServerRescueNegativeTestJSON-1922759309-project] 10.222.0.20 "GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb" status: 404 len: 111 microversion: 2.1 time: 0.032743 Sep 18 06:59:38.356466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1662/2741] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 06:59:38 2022] GET /compute/v2.1/servers/1dd95cf5-d6be-4de9-9941-8b1d35855bcb => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:00:07.275185 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=30,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:07.276017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:07.497969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-56a49352-6157-490e-ad06-7bb3ebf77a75 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Action: 'create', calling method: .version_select at 0x7fe6701f8ca0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-671757144", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMEK/5tNOg0+VFnjjY63v5WNvWuvj/nCHcwDa7JkoDPNcJxnSgOePTS5z8fX39IGyCk9T+krWTrZOeA3cueJV1fPdLqltDfY+oTNUTnln71xJ4W/699LHMzJRQoSU+AO7Q=="}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:07.514329 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-56a49352-6157-490e-ad06-7bb3ebf77a75 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Getting quotas for project f707372e1ec948dcb7e89b9a046cccfc. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 07:00:07.518092 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-56a49352-6157-490e-ad06-7bb3ebf77a75 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Getting quotas for user b997560479c642e0ac43c0b08a1bf157 and project f707372e1ec948dcb7e89b9a046cccfc. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 07:00:07.544144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-56a49352-6157-490e-ad06-7bb3ebf77a75 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.274923 Sep 18 07:00:07.545311 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1080/2742] 10.222.0.20 () {60 vars in 1159 bytes} [Sun Sep 18 07:00:07 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:00:11.204372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:11.204961 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:11.207525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 07:00:11.207675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 07:00:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e (ip 10.222.0.20) !!! Sep 18 07:00:11.208645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=76,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:11.209304 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:11.218260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:11.218753 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:11.219191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:11.219883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:11.220346 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:11.220751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:11.252142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 07:00:11.334300 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d53fae7c-c545-4c7d-9ad9-fe12b3426d54 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 204 len: 0 microversion: 2.1 time: 0.133528 Sep 18 07:00:11.335126 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1663/2743] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 07:00:11 2022] DELETE /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 07:00:11.344359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:11.345099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:11.351154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:11.351627 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:11.352096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:11.352788 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:11.353250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:11.353674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:11.489155 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:11.497905 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-778ede9a-52eb-4bfb-ae2a-f5b1828c9801 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1602 microversion: 2.1 time: 0.158589 Sep 18 07:00:11.498474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1081/2744] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:11 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1602 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:12.015582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:12.016162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:12.021994 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:12.022501 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:12.022978 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:12.023653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:12.024586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:12.025004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:12.055109 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 07:00:12.062330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Working on deleting snapshot 95d4d8a7-fedd-47a6-999f-2e3a7a1b3bf7 from shelved instance... Sep 18 07:00:12.521238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:12.521870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:12.529339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:12.529858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:12.530291 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:12.531025 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:12.538200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:12.538682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:12.678070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:12.686902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c80f25a7-bacf-4de2-918b-05611f50f1c1 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1602 microversion: 2.1 time: 0.169485 Sep 18 07:00:12.687464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1082/2745] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:12 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1602 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:12.797418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Doing local delete in cell 63c4bc6f-f107-41b9-b2c1-7478fed24e9a(cell1) {{(pid=120381) _delete /opt/stack/nova/nova/compute/api.py:2546}} Sep 18 07:00:12.798118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:12.798601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:12.799225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:12.799644 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.compute.api [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] instance is in SHELVED_OFFLOADED state, cleanup the instance's info from database. Sep 18 07:00:12.803454 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] deallocate_for_instance() {{(pid=120381) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1780}} Sep 18 07:00:13.690735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-523905ab-1cf6-4da5-879f-bf693c0324f7 req-14fd999f-5fc9-4183-8fba-820ec5fee46a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ed3cfa3-6093-4c26-9877-01470acd2e16", "name": "network-vif-deleted", "tag": "84893f57-6d56-4075-9e20-5bd22c657e44"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:13.693789 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Updating instance_info_cache with network_info: [] {{(pid=120381) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 18 07:00:13.703643 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-523905ab-1cf6-4da5-879f-bf693c0324f7 req-14fd999f-5fc9-4183-8fba-820ec5fee46a service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:13.704120 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-523905ab-1cf6-4da5-879f-bf693c0324f7 req-14fd999f-5fc9-4183-8fba-820ec5fee46a service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:13.704760 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-523905ab-1cf6-4da5-879f-bf693c0324f7 req-14fd999f-5fc9-4183-8fba-820ec5fee46a service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:13.717469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.compute.server_external_events [req-523905ab-1cf6-4da5-879f-bf693c0324f7 req-14fd999f-5fc9-4183-8fba-820ec5fee46a service nova] Unable to find a host for instance 5ed3cfa3-6093-4c26-9877-01470acd2e16. Dropping event network-vif-deleted {{(pid=120380) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 18 07:00:13.718361 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-523905ab-1cf6-4da5-879f-bf693c0324f7 req-14fd999f-5fc9-4183-8fba-820ec5fee46a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.031752 Sep 18 07:00:13.719477 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1083/2746] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 180 bytes in 33 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Sep 18 07:00:13.725113 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:13.725728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:13.732599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:13.733090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:13.733699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:13.734596 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:13.735071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:13.735614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:13.870894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:13.879513 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-658b32b6-8f51-40a8-abe0-fa936b9263a9 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1602 microversion: 2.1 time: 0.158916 Sep 18 07:00:13.880263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1084/2747] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:13 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1602 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:14.903140 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:14.903757 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:14.910404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:14.910936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:14.911378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:14.912085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:14.912543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:14.912968 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:15.046714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:15.054914 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-85a3b281-4487-46af-bf35-4764698612d0 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1602 microversion: 2.1 time: 0.155625 Sep 18 07:00:15.055474 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1085/2748] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:14 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1602 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:00:15.577161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-31f12904-7452-42ed-8319-fd2cc3f1492a req-5aef4e9d-a113-46e0-bdbb-302548e11fb6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b4bf8fc2-b57b-4198-939c-bd88739c785e", "name": "network-vif-deleted", "tag": "1956a12e-1f36-454a-baa5-f949db978b94"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:15.597438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-31f12904-7452-42ed-8319-fd2cc3f1492a req-5aef4e9d-a113-46e0-bdbb-302548e11fb6 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:15.597921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-31f12904-7452-42ed-8319-fd2cc3f1492a req-5aef4e9d-a113-46e0-bdbb-302548e11fb6 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:15.598789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-31f12904-7452-42ed-8319-fd2cc3f1492a req-5aef4e9d-a113-46e0-bdbb-302548e11fb6 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:15.610730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-31f12904-7452-42ed-8319-fd2cc3f1492a req-5aef4e9d-a113-46e0-bdbb-302548e11fb6 service nova] Creating event network-vif-deleted:1956a12e-1f36-454a-baa5-f949db978b94 for instance b4bf8fc2-b57b-4198-939c-bd88739c785e on hv-cinder-85535 Sep 18 07:00:15.616815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-31f12904-7452-42ed-8319-fd2cc3f1492a req-5aef4e9d-a113-46e0-bdbb-302548e11fb6 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044013 Sep 18 07:00:15.617745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1086/2749] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:00:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:00:16.079375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:16.080129 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:16.088358 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:16.088851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:16.089464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:16.090338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:16.090815 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:16.091239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:16.092579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.scheduler.client.report [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Deleted allocations for instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 Sep 18 07:00:16.132013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:16.132504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:16.132988 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:16.133475 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.objects.instance [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lazy-loading 'fault' on Instance uuid b4bf8fc2-b57b-4198-939c-bd88739c785e {{(pid=120380) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 07:00:16.204374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd862cc69267470580ef23e0223742e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2af8940da97b4298a92e1d45f70cfe29', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:16.214679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0f36ba45-72f2-4c9f-bc69-36820c3a1bd2 tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 200 len: 1484 microversion: 2.1 time: 0.139660 Sep 18 07:00:16.216177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1087/2750] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:16 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 1484 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:16.247227 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-342d9fe0-8571-42b7-9514-0fd75cb699fb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "DELETE /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 204 len: 0 microversion: 2.20 time: 4.235767 Sep 18 07:00:16.248078 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1664/2751] 10.222.0.20 () {60 vars in 1280 bytes} [Sun Sep 18 07:00:12 2022] DELETE /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 0 bytes in 4238 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 18 07:00:16.258675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:16.259299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:16.265273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:16.265783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:16.266221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:16.267193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:16.273104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:16.273104 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:16.283362 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] HTTP exception thrown: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Request method failure captured: Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 HTTP/1.1 Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrn1lB3_UfTsYaV5VDBK-ypfzlj8urkIAwTPw5Yd2WBNRG2q1UHomYg5YU0Q_Gl1eVXe7O3KzhOZQBsmLKDSwprGT0ibDlrMshfvA4R7vtAqET5IlKrB0SXuHk2TN8wUSFWEaynYf04KL8MaFpR88nkIfYDjj1PbzrH7W__S3Z_5VEc Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 07:00:16.285016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 00399347c76d451d94d45f4edc3cf416 Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:00:16.287439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '5ed3cfa3-6093-4c26-9877-01470acd2e16'} Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:00:16.288941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:00:16.290854 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:00:16.292691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Returning 404 to user: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:00:16.292691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-56381bd4-7ee9-40af-90ed-e29d786381c5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 404 len: 111 microversion: 2.20 time: 0.033487 Sep 18 07:00:16.292691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1088/2752] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:16 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 07:00:16.300240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fe8724cf-4e97-4b79-98d6-c971b8d8199a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '.version_select at 0x7fe673b79310>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:16.320099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fe8724cf-4e97-4b79-98d6-c971b8d8199a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1422298950" status: 204 len: 0 microversion: 2.20 time: 0.025274 Sep 18 07:00:16.321204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1665/2753] 10.222.0.20 () {60 vars in 1262 bytes} [Sun Sep 18 07:00:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1422298950 => generated 0 bytes in 27 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 18 07:00:17.095887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-52155e69-8fd8-4707-827a-e38cfd4932e1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'create', calling method: .version_select at 0x7fe6704e2ee0>, body: {"keypair": {"name": "tempest-keypair-84870927", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJSl7d1DIb5U3xAyJWy2sR7rKcmc8jPfZMjMSpAGQ6CP3BtD9+oEZKNS7OpqdOQrLFu0OXxUUDVrMC5znr30Q+lJNL1oQnUTh+Y1W/FxvTLim6VQ0Nq8WLoG9dJpHFZgjg=="}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:17.112932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-52155e69-8fd8-4707-827a-e38cfd4932e1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Getting quotas for project 6fbf7eeb158e4a72bd8133faa1366e0d. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 07:00:17.117000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-52155e69-8fd8-4707-827a-e38cfd4932e1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Getting quotas for user 00399347c76d451d94d45f4edc3cf416 and project 6fbf7eeb158e4a72bd8133faa1366e0d. Resources: {'key_pairs'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 07:00:17.139169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-52155e69-8fd8-4707-827a-e38cfd4932e1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 201 len: 399 microversion: 2.20 time: 0.047109 Sep 18 07:00:17.139872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1089/2754] 10.222.0.20 () {62 vars in 1200 bytes} [Sun Sep 18 07:00:17 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 49 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 18 07:00:17.239976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:17.240713 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] [instance: b4bf8fc2-b57b-4198-939c-bd88739c785e] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:17.248391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:17.249336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:17.250303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:17.251134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:17.251596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:17.252063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:17.265901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] HTTP exception thrown: Instance b4bf8fc2-b57b-4198-939c-bd88739c785e could not be found. Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Request method failure captured: Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e HTTP/1.1 Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrngACylqbekdUBgxiZRNcleTeA1nU-uV5MhYGT7_PJ6j___HFrT5W-YTB0gKFzLUbHlBcdasA-2Gsby7ecSHHQSojsm15W0UL50_BObxG5PsgDsyLZiqOPYOGeRxg_HjcuwxHeeX5pq9-HNdwRSMn_mdcRFuii0mdX8TQJxThw7NYQ Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:00:17.267754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2af8940da97b4298a92e1d45f70cfe29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestMinimumBasicScenario-591471034 Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2af8940da97b4298a92e1d45f70cfe29 Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestMinimumBasicScenario-591471034 Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestMinimumBasicScenario-591471034-project Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: fd862cc69267470580ef23e0223742e0 Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestMinimumBasicScenario-591471034-project Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance b4bf8fc2-b57b-4198-939c-bd88739c785e could not be found. Sep 18 07:00:17.269392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'b4bf8fc2-b57b-4198-939c-bd88739c785e'} Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b4bf8fc2-b57b-4198-939c-bd88739c785e could not be found. Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:00:17.270435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b4bf8fc2-b57b-4198-939c-bd88739c785e could not be found. Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b4bf8fc2-b57b-4198-939c-bd88739c785e could not be found. Sep 18 07:00:17.272223 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:00:17.276804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Returning 404 to user: Instance b4bf8fc2-b57b-4198-939c-bd88739c785e could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:00:17.276804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9c4b164a-8cb8-4d52-b171-6e5976930eca tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e" status: 404 len: 111 microversion: 2.1 time: 0.033754 Sep 18 07:00:17.276804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1666/2755] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:17 2022] GET /compute/v2.1/servers/b4bf8fc2-b57b-4198-939c-bd88739c785e => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:00:17.280118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e5407a51-ea36-43a0-9d0f-d3bdfa1e2a2f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] Calling method '.version_select at 0x7fe670693310>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:17.329601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e5407a51-ea36-43a0-9d0f-d3bdfa1e2a2f tempest-TestMinimumBasicScenario-591471034 tempest-TestMinimumBasicScenario-591471034-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1546923934" status: 202 len: 0 microversion: 2.1 time: 0.054293 Sep 18 07:00:17.330374 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1090/2756] 10.222.0.20 () {58 vars in 1272 bytes} [Sun Sep 18 07:00:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1546923934 => generated 0 bytes in 56 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 07:00:19.297742 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1379025699", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b0343989-7ee1-40ca-87bc-aae149bc4efe"}], "security_groups": [{"name": "tempest-securitygroup--1728790163"}], "key_name": "tempest-keypair-84870927", "user_data": "}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:19.466996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] validate_networks() for [('b0343989-7ee1-40ca-87bc-aae149bc4efe', None, None, None, None, None)] {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 07:00:20.226104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 07:00:20.226629 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 07:00:20.227115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 07:00:20.227907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 07:00:20.228347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 07:00:20.252092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:20.252502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:20.253792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:20.254543 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:20.255046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:20.255485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:20.278663 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Getting quotas for project 6fbf7eeb158e4a72bd8133faa1366e0d. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 07:00:20.283104 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Getting quotas for user 00399347c76d451d94d45f4edc3cf416 and project 6fbf7eeb158e4a72bd8133faa1366e0d. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 07:00:20.295621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 07:00:20.297949 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 07:00:20.357263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bc313262-03b9-4197-87e1-4ccd135e18b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.064047 Sep 18 07:00:20.357907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1667/2757] 10.222.0.20 () {62 vars in 1188 bytes} [Sun Sep 18 07:00:19 2022] POST /compute/v2.1/servers => generated 402 bytes in 1065 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Sep 18 07:00:20.370844 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-405b6c9b-cd9d-4d4d-8c05-94a4103f8b1e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:20.372121 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-405b6c9b-cd9d-4d4d-8c05-94a4103f8b1e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:20.465384 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-405b6c9b-cd9d-4d4d-8c05-94a4103f8b1e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:20.477954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-405b6c9b-cd9d-4d4d-8c05-94a4103f8b1e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:20.502435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-405b6c9b-cd9d-4d4d-8c05-94a4103f8b1e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:20.503753 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-405b6c9b-cd9d-4d4d-8c05-94a4103f8b1e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1280 microversion: 2.20 time: 0.138420 Sep 18 07:00:20.504336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1091/2758] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:20 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1280 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:21.536330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:21.537310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:21.545467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:21.546017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:21.546443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:21.547209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:21.547717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:21.548134 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:21.668727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:21.689354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:21.695653 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:21.700895 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3f00aaaa-ebb0-4b22-b1d5-f0ee5dee9222 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1332 microversion: 2.20 time: 0.169091 Sep 18 07:00:21.702467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1668/2759] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:21 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1332 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:22.730455 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:22.731037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:22.737882 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:22.738400 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:22.738911 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:22.739630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:22.740364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:22.740981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:23.281734 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:23.290298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:23.292111 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:23.293340 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dc4138e8-ff4c-4348-8a13-95ade5621f23 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.567557 Sep 18 07:00:23.293996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1092/2760] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:22 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:24.046725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-4ae2c657-2792-4efc-b146-18826aaf52d0 req-1d735bab-34e4-4cfa-ae8b-97acf2ff75ff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ced1b99-214c-4d5f-957c-e92b0bda948a", "tag": "5d551bba-a366-4a1f-8ba2-e36d636cfc95"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:24.057755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4ae2c657-2792-4efc-b146-18826aaf52d0 req-1d735bab-34e4-4cfa-ae8b-97acf2ff75ff service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:24.058324 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4ae2c657-2792-4efc-b146-18826aaf52d0 req-1d735bab-34e4-4cfa-ae8b-97acf2ff75ff service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:24.059336 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4ae2c657-2792-4efc-b146-18826aaf52d0 req-1d735bab-34e4-4cfa-ae8b-97acf2ff75ff service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:24.072292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-4ae2c657-2792-4efc-b146-18826aaf52d0 req-1d735bab-34e4-4cfa-ae8b-97acf2ff75ff service nova] Creating event network-changed:5d551bba-a366-4a1f-8ba2-e36d636cfc95 for instance 2ced1b99-214c-4d5f-957c-e92b0bda948a on hv-cinder-85535 Sep 18 07:00:24.077559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-4ae2c657-2792-4efc-b146-18826aaf52d0 req-1d735bab-34e4-4cfa-ae8b-97acf2ff75ff service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035118 Sep 18 07:00:24.078370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1669/2761] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:00:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:00:24.319038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:24.319654 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:24.326679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:24.327167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:24.327607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:24.328299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:24.328750 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:24.329174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:24.593489 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:24.602021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:24.603616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:24.604816 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-fcf9f03a-67b7-4bd3-8c8a-983575c93456 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.289921 Sep 18 07:00:24.605449 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1093/2762] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:24 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:25.630967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:25.631932 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:25.638935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:25.639424 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:25.639872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:25.640616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:25.641246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:25.641975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:25.879317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:25.888872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:25.890404 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:25.891852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-95893974-254e-4ef1-b5db-8af4997a0e58 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.265284 Sep 18 07:00:25.892714 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1670/2763] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:25 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:26.917979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:26.919262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:26.926083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:26.926967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:26.927314 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:26.928856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:26.928856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:26.928856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:27.076912 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:27.086324 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:27.087846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:27.089282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6edb8f22-96b5-45bf-ab81-e5ff53228135 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.175823 Sep 18 07:00:27.089838 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1094/2764] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:26 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 07:00:27.569017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-923612697", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-671757144", "security_groups": [{"name": "tempest-secgroup-smoke-1568521079"}], "block_device_mapping_v2": [{"uuid": "18f4a776-d73f-4c69-a733-33b5145cc9bc", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c9fc9740-ace6-449a-b59c-d98e771aa3a8"}]}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:27.708232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] validate_networks() for [('c9fc9740-ace6-449a-b59c-d98e771aa3a8', None, None, None, None, None)] {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 07:00:28.114980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:28.116250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:28.122515 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:28.123037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:28.123471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:28.124167 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:28.124615 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:28.125290 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:28.126327 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 07:00:28.126796 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 07:00:28.127268 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 07:00:28.127883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 07:00:28.128297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 07:00:28.138229 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:28.138719 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:28.139368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:28.140279 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:28.140743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:28.141169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:28.159953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Getting quotas for project f707372e1ec948dcb7e89b9a046cccfc. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 07:00:28.164016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Getting quotas for user b997560479c642e0ac43c0b08a1bf157 and project f707372e1ec948dcb7e89b9a046cccfc. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 07:00:28.229901 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 07:00:28.232286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='18f4a776-d73f-4c69-a733-33b5145cc9bc',volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 07:00:28.302142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:28.310568 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:28.312052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:28.313154 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f0dc4894-b1c4-45be-9389-5a745a71c852 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.202055 Sep 18 07:00:28.313875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1095/2765] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:28 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 07:00:28.380516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5404b783-a13d-41fc-966f-f42e93dbcde5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.814824 Sep 18 07:00:28.381400 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1671/2766] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 07:00:27 2022] POST /compute/v2.1/servers => generated 406 bytes in 817 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 07:00:28.396166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6627144d-37e8-4e70-9f41-0fc5bef0a3da tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:28.399451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6627144d-37e8-4e70-9f41-0fc5bef0a3da tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:28.460922 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6627144d-37e8-4e70-9f41-0fc5bef0a3da tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:28.470052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6627144d-37e8-4e70-9f41-0fc5bef0a3da tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1093 microversion: 2.1 time: 0.080220 Sep 18 07:00:28.471072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1096/2767] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:28 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1093 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:29.337838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:29.338536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:29.344700 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:29.345177 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:29.345606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:29.346495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:29.346964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:29.347389 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:29.495772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:29.503948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:29.505451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:29.514090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:29.514621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:29.515075 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:29.515742 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:29.516465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:29.517102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:29.520188 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:29.521658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:29.526428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dea7f0fe-68d6-4063-b06b-6b20491e4b74 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.192994 Sep 18 07:00:29.539056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1672/2768] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:29 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:29.614561 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:29.626542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2dbbbec6-f31f-4ebc-909d-406f36237cc6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1209 microversion: 2.1 time: 0.129694 Sep 18 07:00:29.627157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1097/2769] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:29 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1209 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:30.563049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:30.563751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:30.571162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:30.571657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:30.572311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:30.573011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:30.573634 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:30.574231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:30.654516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:30.657171 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:30.665783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:30.666487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:30.667310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:30.668334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:30.668976 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:30.669690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:30.716270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:30.724601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:30.726190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:30.727437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ec8cd94c-ec1a-4008-804c-c342cda1f880 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.168310 Sep 18 07:00:30.728082 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1673/2770] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:30 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:30.880133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:30.889174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c0cd1a8f-5ee7-48c6-a1da-b80a340cadbd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.241714 Sep 18 07:00:30.889762 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1098/2771] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:30 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:31.179033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-5a0f23ef-11ae-4022-a1f3-e3ff96718e7f req-530c0c86-e822-497a-a034-8b0358532784 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5cdcc319-ec7c-4d46-a87f-996f83c03821", "tag": "7ea7f43d-b1b2-439b-a869-982bacaa30c8"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:31.190850 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5a0f23ef-11ae-4022-a1f3-e3ff96718e7f req-530c0c86-e822-497a-a034-8b0358532784 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:31.191101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5a0f23ef-11ae-4022-a1f3-e3ff96718e7f req-530c0c86-e822-497a-a034-8b0358532784 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:31.191448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-5a0f23ef-11ae-4022-a1f3-e3ff96718e7f req-530c0c86-e822-497a-a034-8b0358532784 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:31.206189 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-5a0f23ef-11ae-4022-a1f3-e3ff96718e7f req-530c0c86-e822-497a-a034-8b0358532784 service nova] Creating event network-changed:7ea7f43d-b1b2-439b-a869-982bacaa30c8 for instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 on hv-cinder-85535 Sep 18 07:00:31.211151 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-5a0f23ef-11ae-4022-a1f3-e3ff96718e7f req-530c0c86-e822-497a-a034-8b0358532784 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035729 Sep 18 07:00:31.212061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1674/2772] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:00:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:00:31.277808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-33c3ad63-8569-4da5-8f96-ba947440d360 req-8f6683da-ef97-402b-a0f3-50a4349ff177 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ced1b99-214c-4d5f-957c-e92b0bda948a", "name": "network-vif-plugged", "status": "completed", "tag": "5d551bba-a366-4a1f-8ba2-e36d636cfc95"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:31.288284 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-33c3ad63-8569-4da5-8f96-ba947440d360 req-8f6683da-ef97-402b-a0f3-50a4349ff177 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:31.289690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-33c3ad63-8569-4da5-8f96-ba947440d360 req-8f6683da-ef97-402b-a0f3-50a4349ff177 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:31.290323 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-33c3ad63-8569-4da5-8f96-ba947440d360 req-8f6683da-ef97-402b-a0f3-50a4349ff177 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:31.305219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-33c3ad63-8569-4da5-8f96-ba947440d360 req-8f6683da-ef97-402b-a0f3-50a4349ff177 service nova] Creating event network-vif-plugged:5d551bba-a366-4a1f-8ba2-e36d636cfc95 for instance 2ced1b99-214c-4d5f-957c-e92b0bda948a on hv-cinder-85535 Sep 18 07:00:31.311264 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-33c3ad63-8569-4da5-8f96-ba947440d360 req-8f6683da-ef97-402b-a0f3-50a4349ff177 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036931 Sep 18 07:00:31.311887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1099/2773] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:00:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:00:31.756506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:31.757996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:31.764392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:31.764911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:31.765342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:31.766032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:31.766491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:31.766946 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:31.784787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=18,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:DELETE=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:31.896527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:31.904437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:31.905871 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:31.907084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bb4cceea-5a08-4230-bb57-ffec3cfd2026 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.155231 Sep 18 07:00:31.907632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1675/2774] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:31 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:31.913049 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:31.914250 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:31.920412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:31.920901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:31.921342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:31.922030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:31.922488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:31.922967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:32.059959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:32.067528 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-663618ab-58b3-4447-ad71-abc52da8da1a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.159161 Sep 18 07:00:32.068083 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1100/2775] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:31 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:00:32.931536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:32.932150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:32.938507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:32.939026 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:32.939463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:32.940145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:32.940594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:32.941031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:33.085306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:33.089016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:33.089631 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:33.093371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:33.094831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:33.095669 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:33.095930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6c1de50a-5744-4452-bfb6-9a6f88a604ed tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1406 microversion: 2.20 time: 0.168390 Sep 18 07:00:33.096148 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:33.096478 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1676/2776] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:32 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1406 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:33.096605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:33.097239 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:33.097698 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:33.098122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:33.226984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:33.234461 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-74b03ffc-2575-4c80-86c7-a5efc2edacd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.149203 Sep 18 07:00:33.235048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1101/2777] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:33 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:00:34.121449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:34.122014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:34.128681 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:34.129163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:34.129601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:34.130289 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:34.130779 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:34.131206 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:34.257776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:34.258516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:34.259161 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:34.264928 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:34.265413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:34.265868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:34.266536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:34.267006 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:34.267184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:34.267423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:34.268633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:34.269790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-20ba00af-ef2e-4067-888d-cfde83df4d95 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1599 microversion: 2.20 time: 0.152707 Sep 18 07:00:34.270377 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1677/2778] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:00:34 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1599 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:00:34.293929 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:34.294456 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:34.300380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:34.300857 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:34.301294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:34.301980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:34.302436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:34.302882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:34.378210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-bf04b3d7-51db-410f-aa25-5ca3677d28d5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.087879 Sep 18 07:00:34.386658 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1678/2779] 10.222.0.20 () {60 vars in 1316 bytes} [Sun Sep 18 07:00:34 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/os-interface => generated 285 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 18 07:00:34.398764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:34.406367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-20d73049-d206-4e98-a7a9-4aaba62c4238 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.152986 Sep 18 07:00:34.406961 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1102/2780] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:34 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:00:35.430191 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:35.430945 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:35.437313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:35.438060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:35.438503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:35.439227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:35.439674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:35.440281 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:35.490101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-6337ad98-3197-4b07-bbc3-0a5f093cfabf req-d794e1a4-156a-48dd-9f2d-dbacf9c188b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ced1b99-214c-4d5f-957c-e92b0bda948a", "tag": "5d551bba-a366-4a1f-8ba2-e36d636cfc95"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:35.501073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6337ad98-3197-4b07-bbc3-0a5f093cfabf req-d794e1a4-156a-48dd-9f2d-dbacf9c188b9 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:35.501880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6337ad98-3197-4b07-bbc3-0a5f093cfabf req-d794e1a4-156a-48dd-9f2d-dbacf9c188b9 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:35.502437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6337ad98-3197-4b07-bbc3-0a5f093cfabf req-d794e1a4-156a-48dd-9f2d-dbacf9c188b9 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:35.515360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-6337ad98-3197-4b07-bbc3-0a5f093cfabf req-d794e1a4-156a-48dd-9f2d-dbacf9c188b9 service nova] Creating event network-changed:5d551bba-a366-4a1f-8ba2-e36d636cfc95 for instance 2ced1b99-214c-4d5f-957c-e92b0bda948a on hv-cinder-85535 Sep 18 07:00:35.520553 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-6337ad98-3197-4b07-bbc3-0a5f093cfabf req-d794e1a4-156a-48dd-9f2d-dbacf9c188b9 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.232216 Sep 18 07:00:35.521237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1679/2781] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:00:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:00:35.676588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:35.680990 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=100 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:35.685881 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-12cbf107-5c70-475a-a2e3-fa8df0651a22 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.259976 Sep 18 07:00:35.686597 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1103/2782] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:35 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:00:36.710887 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:36.711589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:36.717809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:36.718236 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:36.718705 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:36.719387 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:36.719837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:36.720263 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:36.851143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:36.858738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-cdb2499c-cd2d-47db-8942-28c82f51c8c1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.153223 Sep 18 07:00:36.859294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1680/2783] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:36 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:37.883253 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:37.883861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:37.886147 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=74,nova_api:INSERT=2,nova_api:DELETE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:37.891348 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:37.891837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:37.892275 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:37.892966 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:37.893419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:37.893842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:38.025280 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:38.032852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7d3586d8-6853-48d3-88ed-936f0955fd13 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.154324 Sep 18 07:00:38.033413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1104/2784] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:37 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:39.055323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:39.055866 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:39.058192 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:39.058909 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=12 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:39.064693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:39.065180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:39.065612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:39.066290 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:39.066840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:39.067264 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:39.201053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:39.209603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f6a10a96-a66f-44e6-b0af-ccf26c4305ca tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.158141 Sep 18 07:00:39.210724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1681/2785] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:39 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:40.237227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:40.237852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:40.239779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=12 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:40.245893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:40.246407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:40.246868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:40.247554 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:40.248004 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:40.248425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:40.384315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:40.391994 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-6fd5824f-ce45-4289-b37e-78fc5ff96484 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.158608 Sep 18 07:00:40.392554 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1105/2786] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:40 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:41.415982 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:41.416601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:41.418903 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=78,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=6 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:41.424056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:41.424537 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:41.424964 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:41.425640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:41.426086 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:41.426505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:41.562443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:41.570831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5cfd206b-5859-43f8-bb9c-7509679bec0f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.159245 Sep 18 07:00:41.571411 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1682/2787] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:41 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:42.594849 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:42.595418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:42.601898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:42.602379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:42.602839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:42.603524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:42.603981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:42.604402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:42.734278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:42.742471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-27f1a1a0-3bf5-4c1c-a16f-5c5170282fd5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.151663 Sep 18 07:00:42.743058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1106/2788] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:42 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:43.367006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-4000aaf5-5d91-4b05-8460-7c0a891c2ca2 req-0a55d68e-f933-4d7c-830f-a156dac02088 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5cdcc319-ec7c-4d46-a87f-996f83c03821", "name": "network-vif-plugged", "status": "completed", "tag": "7ea7f43d-b1b2-439b-a869-982bacaa30c8"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:43.376878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4000aaf5-5d91-4b05-8460-7c0a891c2ca2 req-0a55d68e-f933-4d7c-830f-a156dac02088 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:43.377583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4000aaf5-5d91-4b05-8460-7c0a891c2ca2 req-0a55d68e-f933-4d7c-830f-a156dac02088 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:43.378009 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4000aaf5-5d91-4b05-8460-7c0a891c2ca2 req-0a55d68e-f933-4d7c-830f-a156dac02088 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:43.396247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-4000aaf5-5d91-4b05-8460-7c0a891c2ca2 req-0a55d68e-f933-4d7c-830f-a156dac02088 service nova] Creating event network-vif-plugged:7ea7f43d-b1b2-439b-a869-982bacaa30c8 for instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 on hv-cinder-85535 Sep 18 07:00:43.401156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-4000aaf5-5d91-4b05-8460-7c0a891c2ca2 req-0a55d68e-f933-4d7c-830f-a156dac02088 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100812 Sep 18 07:00:43.401745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1683/2789] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:00:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:00:43.766285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:43.766927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:43.773745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:43.774230 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:43.774700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:43.775387 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:43.775842 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:43.776269 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:43.905184 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:43.912872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-dd4f806e-763a-4b1e-b72c-08fd25961220 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1265 microversion: 2.1 time: 0.150613 Sep 18 07:00:43.913437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1107/2790] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:43 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:00:44.937077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:44.937781 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:44.943958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:44.944437 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:44.944875 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:44.945602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:44.946058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:44.946483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:45.079036 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:45.087496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fd5d24fa-324b-4b2b-b260-6578a433cc99 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1452 microversion: 2.1 time: 0.155206 Sep 18 07:00:45.088063 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1684/2791] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:44 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1452 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:45.109895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:00:45.110501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:00:45.116821 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:45.117300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:45.117730 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:45.118402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:45.118871 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:45.119289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:45.249784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:00:45.257381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ddf6b30b-7446-4f04-bab3-76e4c3d9120c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1452 microversion: 2.1 time: 0.151274 Sep 18 07:00:45.257941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1108/2792] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:00:45 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1452 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:00:47.349423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-fe192371-b2a7-4cba-b676-3f52d07fa016 req-28006a33-aa4e-4b39-9bba-d025c6e8dd25 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5cdcc319-ec7c-4d46-a87f-996f83c03821", "tag": "7ea7f43d-b1b2-439b-a869-982bacaa30c8"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:00:47.359240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fe192371-b2a7-4cba-b676-3f52d07fa016 req-28006a33-aa4e-4b39-9bba-d025c6e8dd25 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:00:47.359712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fe192371-b2a7-4cba-b676-3f52d07fa016 req-28006a33-aa4e-4b39-9bba-d025c6e8dd25 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:00:47.360137 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-fe192371-b2a7-4cba-b676-3f52d07fa016 req-28006a33-aa4e-4b39-9bba-d025c6e8dd25 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:00:47.371449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-fe192371-b2a7-4cba-b676-3f52d07fa016 req-28006a33-aa4e-4b39-9bba-d025c6e8dd25 service nova] Creating event network-changed:7ea7f43d-b1b2-439b-a869-982bacaa30c8 for instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 on hv-cinder-85535 Sep 18 07:00:47.376486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-fe192371-b2a7-4cba-b676-3f52d07fa016 req-28006a33-aa4e-4b39-9bba-d025c6e8dd25 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030762 Sep 18 07:00:47.377120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1685/2793] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:00:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:01:11.378442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:11.379054 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:11.382237 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=9 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:11.382951 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=30 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:11.389391 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:11.389883 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:11.390352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:11.391087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:11.391543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:11.391970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:11.416458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c257d372-b47b-4f4d-9c55-7efc5662d808 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/ips" status: 200 len: 153 microversion: 2.20 time: 0.042210 Sep 18 07:01:11.417028 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1109/2794] 10.222.0.20 () {60 vars in 1289 bytes} [Sun Sep 18 07:01:11 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/ips => generated 153 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 07:01:13.311432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=7 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:13.312099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=48 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:13.317007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:01:13.317627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:13.324451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:13.324926 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:13.325362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:13.326041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:13.326494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:13.326941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:13.348725 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lazy-loading 'flavor' on Instance uuid 2ced1b99-214c-4d5f-957c-e92b0bda948a {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 07:01:13.533919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-82d1a49c-7cf1-4faa-8f05-fdf433ddd6f4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/action" status: 202 len: 0 microversion: 2.20 time: 0.224066 Sep 18 07:01:13.534594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1686/2795] 10.222.0.20 () {62 vars in 1319 bytes} [Sun Sep 18 07:01:13 2022] POST /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/action => generated 0 bytes in 226 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 07:01:13.543412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:13.544061 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:13.559308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:13.559308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:13.559308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:13.559308 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:13.560062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:13.560062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:13.684034 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:13.692037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:13.693497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:13.694670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d52ac77e-bfda-47a5-a835-eadc33f8baa4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.155909 Sep 18 07:01:13.695272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1110/2796] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:13 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:14.721046 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:14.721657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:14.729254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:14.729722 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:14.730150 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:14.730854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:14.731308 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:14.731726 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:14.860930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:14.869582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:14.871065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:14.872222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-572d040b-524e-4554-9fef-9e3edc1e4dc8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.156004 Sep 18 07:01:14.872790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1687/2797] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:14 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:15.897316 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:15.897926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:15.904319 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:15.904799 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:15.905231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:15.905909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:15.906352 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:15.906800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:16.038031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:16.046524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:16.047995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:16.049152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ac96c692-5b74-44cc-b13d-b495a202724c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.155753 Sep 18 07:01:16.049708 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1111/2798] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:15 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:17.075032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:17.075665 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:17.082071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:17.082546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:17.083011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:17.083694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:17.084146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:17.084569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:17.209477 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:17.217380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:17.218852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:17.220011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-3a2532b8-7589-43e6-9623-080c226db1a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.148921 Sep 18 07:01:17.220573 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1688/2799] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:17 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:18.244763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:18.245373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:18.251907 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:18.252381 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:18.252810 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:18.253516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:18.253962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:18.254378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:18.383970 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:18.392338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:18.393778 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:18.394962 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-38105280-e1d0-4419-b220-6b8eb887734f tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.154156 Sep 18 07:01:18.395506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1112/2800] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:18 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:19.420170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:19.420868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:19.427035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:19.427512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:19.427980 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:19.428659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:19.429159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:19.429590 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:19.574232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:19.583396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:19.584868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:19.586990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34257c45-d791-4aee-ad5c-4d0b180c56cb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.170318 Sep 18 07:01:19.588814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1689/2801] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:19 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:20.611445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:20.612050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:20.618219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:20.618726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:20.619157 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:20.619830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:20.620282 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:20.620694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:20.745705 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:20.753927 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:20.755402 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:20.756557 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-73ece15a-2d30-4526-8a07-a01842accbeb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.149009 Sep 18 07:01:20.757608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1113/2802] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:20 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:21.782746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:21.783358 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:21.790351 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:21.790868 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:21.791305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:21.791986 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:21.792468 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:21.792891 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:21.918064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:21.926069 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:21.927558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:21.928699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-dacbca4e-b264-4af5-92e9-4cf6c83facd8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.149927 Sep 18 07:01:21.929254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1690/2803] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:21 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:22.953533 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:22.954142 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:22.961437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:22.961926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:22.962364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:22.963085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:22.963537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:22.963963 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:23.092417 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:23.100359 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:23.101855 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:23.103037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e15a8db1-4ae8-46f6-b657-804b360b894d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.153366 Sep 18 07:01:23.103585 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1114/2804] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:22 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:24.128436 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:24.129051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:24.135585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:24.136067 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:24.136503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:24.137185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:24.137635 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:24.138055 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:24.263017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:24.271051 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:24.273092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:24.274247 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f2f5ab83-7be1-495f-821b-041294a9e1dd tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.149801 Sep 18 07:01:24.274865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1691/2805] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:24 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:25.298084 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:25.298749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:25.305825 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:25.306310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:25.306789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:25.307482 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:25.307937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:25.308366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:25.435884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:25.443851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:25.445310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:25.446459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e2072923-2305-48c3-bece-de49e7bef633 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.152294 Sep 18 07:01:25.447035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1115/2806] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:25 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:26.471776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:26.472388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:26.478941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:26.479421 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:26.479860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:26.480542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:26.480996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:26.481417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:26.610445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:26.619034 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:26.620476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:26.621621 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ccc775dd-8626-40ac-81eb-af0e741ca609 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.153745 Sep 18 07:01:26.622184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1692/2807] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:26 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:27.646719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:27.647931 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:27.654620 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:27.655109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:27.655546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:27.656252 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:27.656703 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:27.657135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:27.788599 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:27.796660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:27.798105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:27.799301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d3a1a8e3-b988-49c4-be11-e9011b479833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.156537 Sep 18 07:01:27.799862 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1116/2808] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:27 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:28.824970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:28.825547 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:28.833241 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:28.834084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:28.835555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:28.836797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:28.837599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:28.838352 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:28.968867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:28.977001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:28.978445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:28.979618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ff43916c-d3e6-412c-88ea-cd902e7b055b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.158734 Sep 18 07:01:28.980180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1693/2809] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:28 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:30.004107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:30.004721 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:30.011297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:30.011779 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:30.012209 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:30.012892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:30.013345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:30.013768 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:30.138793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:30.146996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:30.148490 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:30.149672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9b41f852-5f23-4a25-8160-08a22073a225 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1745 microversion: 2.20 time: 0.149441 Sep 18 07:01:30.150231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1117/2810] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:29 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1745 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:31.175243 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:31.176010 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:31.185343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:31.186195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:31.186837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:31.187617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:31.188703 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:31.189620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:31.328691 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:31.339366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:31.340380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:31.341661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f160dee1-98cc-4c35-af4c-9f895781fe5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1740 microversion: 2.20 time: 0.170824 Sep 18 07:01:31.342257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1694/2811] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:31 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1740 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:32.373227 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:32.374060 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:32.380318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:32.381065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:32.381941 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:32.382794 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:32.383800 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:32.384484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:32.529273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:32.537996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:32.539504 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:32.540880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0d83a20b-2c08-46f9-819e-ccd9e2b6b833 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1740 microversion: 2.20 time: 0.176260 Sep 18 07:01:32.541451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1118/2812] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:32 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1740 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:33.567739 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:33.569666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:33.576867 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:33.577612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:33.578221 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:33.581815 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:33.582534 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:33.583186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:33.642714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-25957459-5e2a-4a9c-a8d4-b1504fe0f1ae req-e2d6049f-c1b0-4dd0-bfb2-0cef905ef4d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ced1b99-214c-4d5f-957c-e92b0bda948a", "name": "network-vif-plugged", "status": "completed", "tag": "5d551bba-a366-4a1f-8ba2-e36d636cfc95"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:01:33.666278 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-25957459-5e2a-4a9c-a8d4-b1504fe0f1ae req-e2d6049f-c1b0-4dd0-bfb2-0cef905ef4d8 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:33.667133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-25957459-5e2a-4a9c-a8d4-b1504fe0f1ae req-e2d6049f-c1b0-4dd0-bfb2-0cef905ef4d8 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:33.667867 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-25957459-5e2a-4a9c-a8d4-b1504fe0f1ae req-e2d6049f-c1b0-4dd0-bfb2-0cef905ef4d8 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:33.695124 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-25957459-5e2a-4a9c-a8d4-b1504fe0f1ae req-e2d6049f-c1b0-4dd0-bfb2-0cef905ef4d8 service nova] Creating event network-vif-plugged:5d551bba-a366-4a1f-8ba2-e36d636cfc95 for instance 2ced1b99-214c-4d5f-957c-e92b0bda948a on hv-cinder-85535 Sep 18 07:01:33.701702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-25957459-5e2a-4a9c-a8d4-b1504fe0f1ae req-e2d6049f-c1b0-4dd0-bfb2-0cef905ef4d8 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064224 Sep 18 07:01:33.702735 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1119/2813] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:01:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:01:33.741184 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:33.750347 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:33.751831 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:33.753003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7917112c-e5c6-4d6b-87d6-af87261bb278 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1740 microversion: 2.20 time: 0.189088 Sep 18 07:01:33.753569 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1695/2814] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:33 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1740 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:34.780287 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:34.781138 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:34.789943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:34.790516 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:34.791097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:34.791857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:34.792315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:34.792975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:34.933512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:34.943214 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:34.945156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:34.946330 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a2d12c31-da34-4b92-bae2-926beb5f16a3 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1740 microversion: 2.20 time: 0.170862 Sep 18 07:01:34.954203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1120/2815] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:34 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1740 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:35.973194 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:35.973820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:35.983091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:35.983599 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:35.984245 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:35.985191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:35.985688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:35.986154 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:36.140464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:36.149467 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:36.151066 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:36.152165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d96f3934-0bf5-4cb7-9a1d-f77ef939b149 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1740 microversion: 2.20 time: 0.184045 Sep 18 07:01:36.152747 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1696/2816] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:35 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1740 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:37.177860 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:37.179152 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:37.194298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:37.195172 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:37.195672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:37.196584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:37.197035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:37.197459 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:37.344776 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:37.354110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:37.355591 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:37.356771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c3daf8df-bda6-4215-b054-3d948b0bfc40 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1740 microversion: 2.20 time: 0.182840 Sep 18 07:01:37.370820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1121/2817] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:37 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1740 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:38.382962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:38.383535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:38.392310 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:38.393166 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:38.393771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:38.394466 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:38.394944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:38.395368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:38.548091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:38.558059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:38.559589 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:38.560770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6bbca79e-35db-4a4f-b3de-cc2886684b3e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1740 microversion: 2.20 time: 0.182216 Sep 18 07:01:38.561339 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1697/2818] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:38 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1740 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:39.585903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:39.586526 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:39.600595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:39.601483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:39.602190 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:39.603195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:39.603736 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:39.606807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:39.757786 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:39.767053 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:39.768488 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:39.769697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a4116d05-c04d-487b-ba25-95b093fab8b2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1740 microversion: 2.20 time: 0.188353 Sep 18 07:01:39.770241 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1122/2819] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:39 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1740 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:40.797651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:40.798571 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:40.805974 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:40.806459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:40.806930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:40.807633 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:40.808087 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:40.808513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:40.943170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:40.951463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:40.953028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:40.954371 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-308d7cac-4aff-46d4-8ca1-4106457a046d tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1722 microversion: 2.20 time: 0.162006 Sep 18 07:01:40.954972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1698/2820] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:40 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1722 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 07:01:41.979591 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:41.980238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:41.982679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=29 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:41.988085 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:41.988574 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:41.989017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:41.989710 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:41.990680 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:41.991119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:41.995174 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=79 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:42.117650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:42.125975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:42.127458 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:42.128619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f2043373-3e11-41b7-b700-48de6e423e0e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1722 microversion: 2.20 time: 0.152974 Sep 18 07:01:42.129176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1123/2821] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:41 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1722 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:43.154249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:43.154882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:43.161297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:43.161778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:43.162208 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:43.162941 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:43.163393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:43.163814 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:43.294288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:43.302286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:43.303737 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:43.304911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0bb6fcf6-29dd-44a9-8745-f126d0555238 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1725 microversion: 2.20 time: 0.154550 Sep 18 07:01:43.305465 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1699/2822] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:43 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:43.329365 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4345a7eb-acb2-44f3-b21f-2b40d25d19c4"}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:01:43.331689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:43.338713 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:43.339192 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:43.339622 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:43.340297 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:43.340743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:43.341161 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:43.614312 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f4122ce1-1230-478a-b13f-fb215774e5ab tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.288829 Sep 18 07:01:43.614947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1124/2823] 10.222.0.20 () {62 vars in 1364 bytes} [Sun Sep 18 07:01:43 2022] POST /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/os-volume_attachments => generated 188 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 07:01:43.743545 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:43.744039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:43.745976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=29 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:43.751535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:43.752016 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:43.752451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:43.753128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:43.753581 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:43.754003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:43.757068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:43.987147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-da561a16-0c27-444b-90c8-99f856d749e8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "DELETE /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/os-volume_attachments/4345a7eb-acb2-44f3-b21f-2b40d25d19c4" status: 202 len: 0 microversion: 2.20 time: 0.247993 Sep 18 07:01:43.987734 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1700/2824] 10.222.0.20 () {60 vars in 1457 bytes} [Sun Sep 18 07:01:43 2022] DELETE /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/os-volume_attachments/4345a7eb-acb2-44f3-b21f-2b40d25d19c4 => generated 0 bytes in 249 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 07:01:44.047429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:01:44.047967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:44.054592 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:44.055100 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:44.055535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:44.056222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:44.056676 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:44.057103 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:44.144399 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120380) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 18 07:01:44.210609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-67d93c63-cf72-457c-a3ef-30ee1a6dcb8b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "POST /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/action" status: 202 len: 0 microversion: 2.20 time: 0.167037 Sep 18 07:01:44.211285 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1125/2825] 10.222.0.20 () {62 vars in 1319 bytes} [Sun Sep 18 07:01:44 2022] POST /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/action => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 07:01:44.219224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:44.219915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:44.227860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:44.228343 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:44.228780 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:44.229463 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:44.229915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:44.230338 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:44.373701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:44.382190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:44.383748 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:44.384943 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-58012c67-cedc-44b7-afed-4a51a461bc9e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1673 microversion: 2.20 time: 0.169572 Sep 18 07:01:44.385532 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1701/2826] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:44 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1673 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:45.410131 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:45.410804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:45.417647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:45.418136 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:45.418595 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:45.419295 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:45.419754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:45.420181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:45.561185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-a77f0e8e-3c1b-4a54-b523-b2cf42ef8168 req-d6f4284a-944c-4c9e-8b30-a699fd3b4cc0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ced1b99-214c-4d5f-957c-e92b0bda948a", "name": "network-vif-unplugged", "status": "completed", "tag": "5d551bba-a366-4a1f-8ba2-e36d636cfc95"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:01:45.571958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-a77f0e8e-3c1b-4a54-b523-b2cf42ef8168 req-d6f4284a-944c-4c9e-8b30-a699fd3b4cc0 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:45.572915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-a77f0e8e-3c1b-4a54-b523-b2cf42ef8168 req-d6f4284a-944c-4c9e-8b30-a699fd3b4cc0 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:45.573592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-a77f0e8e-3c1b-4a54-b523-b2cf42ef8168 req-d6f4284a-944c-4c9e-8b30-a699fd3b4cc0 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:45.587723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-a77f0e8e-3c1b-4a54-b523-b2cf42ef8168 req-d6f4284a-944c-4c9e-8b30-a699fd3b4cc0 service nova] Creating event network-vif-unplugged:5d551bba-a366-4a1f-8ba2-e36d636cfc95 for instance 2ced1b99-214c-4d5f-957c-e92b0bda948a on hv-cinder-85535 Sep 18 07:01:45.594772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-a77f0e8e-3c1b-4a54-b523-b2cf42ef8168 req-d6f4284a-944c-4c9e-8b30-a699fd3b4cc0 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037382 Sep 18 07:01:45.596085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1702/2827] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:01:45 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:01:45.618198 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:45.627445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:45.629046 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:45.630199 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-4e41e264-ed35-47f2-aacc-32dd3ce509a4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.223958 Sep 18 07:01:45.631226 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1126/2828] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:45 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:46.656701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:46.657428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:46.674506 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:46.675366 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:46.675803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:46.676504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:46.677115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:46.677574 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:46.830764 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:46.840960 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:46.842431 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:46.843797 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-60f105e7-a9a8-4a0b-bc8e-2551915203d2 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.191505 Sep 18 07:01:46.844998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1703/2829] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:46 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:47.873983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:47.874570 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:47.881902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:47.882390 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:47.882869 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:47.883567 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:47.884019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:47.884440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:48.022271 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:48.049624 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:48.053536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:48.053536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ea149c58-9b30-4c23-bf4a-e11a8b6e87d6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.183044 Sep 18 07:01:48.054050 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1127/2830] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:47 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:49.079106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:49.079894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:49.086656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:49.087146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:49.087588 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:49.088291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:49.088749 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:49.089179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:49.236816 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:49.246120 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:49.247767 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:49.249210 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-25680e02-0e90-4002-8aee-9b04b36e7668 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.174065 Sep 18 07:01:49.249771 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1704/2831] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:49 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:50.280607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:50.281386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:50.288598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:50.289423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:50.289863 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:50.290560 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:50.291037 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:50.291665 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:50.440058 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:50.449081 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:50.450484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:50.458763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-04789c12-bdbf-48a4-a345-157c50019554 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.178527 Sep 18 07:01:50.470783 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1128/2832] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:50 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:51.483567 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:51.484260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:51.491499 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:51.491990 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:51.492435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:51.493201 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:51.493654 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:51.494076 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:51.639556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:51.649045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:51.650672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:51.651933 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-14ee9806-b5d7-4849-b908-96f954374e5b tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.172332 Sep 18 07:01:51.652777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1705/2833] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:51 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:52.678043 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:52.679447 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:52.686796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:52.687310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:52.687748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:52.688446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:52.688909 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:52.689334 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:52.824392 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:52.832804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:52.834857 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:52.836072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1b634e83-128b-4b81-886b-04a4a3c151bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.162722 Sep 18 07:01:52.836811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1129/2834] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:52 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:53.861059 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:53.861708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:53.868865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:53.869356 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:53.869792 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:53.870494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:53.870971 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:53.871395 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:54.008701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:54.016999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:54.018443 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:54.019642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-aeece3a5-b400-41c2-be7b-f2765b528700 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.162893 Sep 18 07:01:54.020197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1706/2835] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:53 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:55.044213 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:55.044830 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:55.051436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:55.051920 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:55.052363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:55.053048 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:55.053518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:55.053929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:55.186535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:55.195203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:55.196623 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:55.197765 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-df56e00b-6ec9-46d4-b638-9d9f72bd7e85 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.157536 Sep 18 07:01:55.198313 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1130/2836] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:55 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:56.223106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:56.223709 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:56.230143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:56.230656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:56.231093 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:56.231776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:56.232226 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:56.232650 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:56.358544 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:56.367090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:56.368533 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:56.369692 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-2ce38fbf-5865-4907-ae2d-7ac80a17bae6 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.150509 Sep 18 07:01:56.370249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1707/2837] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:56 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:57.395246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:57.395858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:57.402837 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:57.403339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:57.403758 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:57.404442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:57.404895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:57.405318 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:57.533865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:57.541924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:57.543398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:57.544562 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e8a6d07b-754c-4313-9a0c-24a08eda00b5 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.153247 Sep 18 07:01:57.545126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1131/2838] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:57 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:58.571142 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:58.573380 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:58.577953 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:58.579269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:58.579269 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:58.579579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:58.580039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:58.580039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:58.727075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:58.735362 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:58.736822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:58.738335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-af9510e9-8bc0-4cd7-acc8-346b6d385a98 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.172010 Sep 18 07:01:58.738930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1708/2839] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:58 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:01:59.642335 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-d4fdc91b-cf08-40e2-bf9a-b70defb8c9c1 req-d48af285-8c8e-42df-a9d1-69f4af00e1df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ced1b99-214c-4d5f-957c-e92b0bda948a", "name": "network-vif-plugged", "status": "completed", "tag": "5d551bba-a366-4a1f-8ba2-e36d636cfc95"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:01:59.654353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d4fdc91b-cf08-40e2-bf9a-b70defb8c9c1 req-d48af285-8c8e-42df-a9d1-69f4af00e1df service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:59.655701 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d4fdc91b-cf08-40e2-bf9a-b70defb8c9c1 req-d48af285-8c8e-42df-a9d1-69f4af00e1df service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:59.656166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-d4fdc91b-cf08-40e2-bf9a-b70defb8c9c1 req-d48af285-8c8e-42df-a9d1-69f4af00e1df service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:59.669937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-d4fdc91b-cf08-40e2-bf9a-b70defb8c9c1 req-d48af285-8c8e-42df-a9d1-69f4af00e1df service nova] Creating event network-vif-plugged:5d551bba-a366-4a1f-8ba2-e36d636cfc95 for instance 2ced1b99-214c-4d5f-957c-e92b0bda948a on hv-cinder-85535 Sep 18 07:01:59.675102 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-d4fdc91b-cf08-40e2-bf9a-b70defb8c9c1 req-d48af285-8c8e-42df-a9d1-69f4af00e1df service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036511 Sep 18 07:01:59.675731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1132/2840] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:01:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:01:59.764593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:01:59.765209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:01:59.771893 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:59.772388 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:59.772826 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:59.773519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:01:59.773970 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:01:59.774393 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:01:59.908345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:59.917270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:59.918751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:01:59.919927 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-4d77a64d-5fe6-4026-8eaa-45ea4dd3e4b4 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.159718 Sep 18 07:01:59.920494 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1709/2841] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:01:59 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 07:02:00.945861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:00.946629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:00.952817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:00.953353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:00.953795 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:00.954491 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:00.954989 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:00.955418 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:01.096412 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:01.105022 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:01.106548 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:01.107729 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f1974654-0806-429f-aa47-9c561acce2df tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1731 microversion: 2.20 time: 0.167221 Sep 18 07:02:01.108339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1133/2842] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:02:00 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1731 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:02:02.134049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:02.134620 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:02.140996 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:02.141479 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:02.141919 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:02.142631 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:02.143089 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:02.143516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:02.269303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:02.277363 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:02.278843 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:02.279993 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-0c04499a-774e-4a1d-9dfd-b4fac845a1ff tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1718 microversion: 2.20 time: 0.150537 Sep 18 07:02:02.280743 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1710/2843] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:02:02 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1718 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:02:33.868408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:33.870553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:33.873118 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=25,nova_api:UPDATE=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:33.873773 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=74,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:33.880774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:33.881254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:33.881690 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:33.882379 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:33.882939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:33.883367 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:33.913517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0b1a58ca-5e3d-4961-a988-511f678c7df0 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.049485 Sep 18 07:02:33.914070 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1134/2844] 10.222.0.20 () {60 vars in 1343 bytes} [Sun Sep 18 07:02:33 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:02:33.976872 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:33.977450 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:33.979492 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=21 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:33.980175 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=70,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:33.985666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:33.986146 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:33.986601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:33.987320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:33.987772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:33.988203 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:34.014012 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 07:02:34.088110 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6bc51ca2-c17f-465c-92d5-80d567847978 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "DELETE /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 204 len: 0 microversion: 2.20 time: 0.115609 Sep 18 07:02:34.088701 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1711/2845] 10.222.0.20 () {60 vars in 1280 bytes} [Sun Sep 18 07:02:33 2022] DELETE /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 18 07:02:34.099546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:34.100737 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:34.106774 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:34.107260 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:34.107702 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:34.108394 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:34.108850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:34.109273 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:34.284605 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:34.292524 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:34.293935 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:34.295110 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-b61cc10d-895e-4c1d-9f46-19066036eed8 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.201468 Sep 18 07:02:34.295666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1135/2846] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:02:34 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:02:35.321617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:35.322306 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:35.328784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:35.329257 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:35.329693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:35.330370 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:35.330838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:35.331262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:35.521360 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:35.529420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:35.530874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:35.532031 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-8b5dca4d-3676-4028-8f97-91c5c336ac78 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1724 microversion: 2.20 time: 0.214746 Sep 18 07:02:35.532587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1712/2847] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:02:35 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1724 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:02:36.273327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-ec212391-5b6b-4745-87ca-7e07fb95d74d req-b8b12d62-6bc9-4d44-bfe5-92a87aa124c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ced1b99-214c-4d5f-957c-e92b0bda948a", "name": "network-vif-deleted", "tag": "5d551bba-a366-4a1f-8ba2-e36d636cfc95"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:02:36.283506 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ec212391-5b6b-4745-87ca-7e07fb95d74d req-b8b12d62-6bc9-4d44-bfe5-92a87aa124c9 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:36.284266 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ec212391-5b6b-4745-87ca-7e07fb95d74d req-b8b12d62-6bc9-4d44-bfe5-92a87aa124c9 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:36.284882 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ec212391-5b6b-4745-87ca-7e07fb95d74d req-b8b12d62-6bc9-4d44-bfe5-92a87aa124c9 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:36.297440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-ec212391-5b6b-4745-87ca-7e07fb95d74d req-b8b12d62-6bc9-4d44-bfe5-92a87aa124c9 service nova] Creating event network-vif-deleted:5d551bba-a366-4a1f-8ba2-e36d636cfc95 for instance 2ced1b99-214c-4d5f-957c-e92b0bda948a on hv-cinder-85535 Sep 18 07:02:36.303537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-ec212391-5b6b-4745-87ca-7e07fb95d74d req-b8b12d62-6bc9-4d44-bfe5-92a87aa124c9 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033797 Sep 18 07:02:36.304144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1136/2848] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:02:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:02:36.557267 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:36.557894 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:36.564231 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:36.564712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:36.565152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:36.565839 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:36.566291 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:36.567259 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:36.598213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:36.598660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:36.599106 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:36.599550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lazy-loading 'fault' on Instance uuid 2ced1b99-214c-4d5f-957c-e92b0bda948a {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 07:02:36.641333 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:36.649513 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:36.650999 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '00399347c76d451d94d45f4edc3cf416', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6fbf7eeb158e4a72bd8133faa1366e0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:36.652157 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-7be81f9b-4855-4294-94fd-56db003988bf tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 200 len: 1370 microversion: 2.20 time: 0.098808 Sep 18 07:02:36.652721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1713/2849] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:02:36 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 1370 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 07:02:37.678090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:37.678876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:37.685149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:37.685648 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:37.686151 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:37.686915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:37.687502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:37.687832 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:37.701789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] HTTP exception thrown: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Request method failure captured: Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a HTTP/1.1 Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrn1lB3_UfTsYaV5VDBK-ypfzlj8urkIAwTPw5Yd2WBNRG2q1UHomYg5YU0Q_Gl1eVXe7O3KzhOZQBsmLKDSwprGT0ibDlrMshfvA4R7vtAqET5IlKrB0SXuHk2TN8wUSFWEaynYf04KL8MaFpR88nkIfYDjj1PbzrH7W__S3Z_5VEc Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 07:02:37.703473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 00399347c76d451d94d45f4edc3cf416 Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:37.705422 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '2ced1b99-214c-4d5f-957c-e92b0bda948a'} Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:02:37.706465 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:37.708740 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:37.710219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Returning 404 to user: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:02:37.710219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d4a4d78b-53d3-4e22-ba93-e7b676d35a1a tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 404 len: 111 microversion: 2.20 time: 0.031551 Sep 18 07:02:37.710219 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1137/2850] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:02:37 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 07:02:37.714103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-01460383-e8e1-4b68-bd55-034678b8ec2c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '.version_select at 0x7fe6701bbd30>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:37.718669 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR oslo.messaging._drivers.impl_rabbit [None req-01460383-e8e1-4b68-bd55-034678b8ec2c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [1e328eaa-c087-42fa-bb3a-81f1d238ae95] AMQP server on 10.222.0.20:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 18 07:02:38.726540 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [None req-01460383-e8e1-4b68-bd55-034678b8ec2c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [1e328eaa-c087-42fa-bb3a-81f1d238ae95] Reconnected to AMQP server on 10.222.0.20:5672 via [amqp] client with port 52822. Sep 18 07:02:38.748311 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-01460383-e8e1-4b68-bd55-034678b8ec2c tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-84870927" status: 204 len: 0 microversion: 2.20 time: 1.037915 Sep 18 07:02:38.748882 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1714/2851] 10.222.0.20 () {60 vars in 1256 bytes} [Sun Sep 18 07:02:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-84870927 => generated 0 bytes in 1039 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 18 07:02:40.720947 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:40.721456 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:40.728378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:40.728864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:40.729301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:40.729984 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:40.730436 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:40.730892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:40.743987 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] HTTP exception thrown: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Request method failure captured: Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a HTTP/1.1 Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrn1lB3_UfTsYaV5VDBK-ypfzlj8urkIAwTPw5Yd2WBNRG2q1UHomYg5YU0Q_Gl1eVXe7O3KzhOZQBsmLKDSwprGT0ibDlrMshfvA4R7vtAqET5IlKrB0SXuHk2TN8wUSFWEaynYf04KL8MaFpR88nkIfYDjj1PbzrH7W__S3Z_5VEc Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 07:02:40.745539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 00399347c76d451d94d45f4edc3cf416 Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.747672 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '2ced1b99-214c-4d5f-957c-e92b0bda948a'} Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:02:40.748671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.750300 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.752073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Returning 404 to user: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:02:40.752073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-77a79053-9099-47f8-b73e-83662b7ec1bb tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "DELETE /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 404 len: 111 microversion: 2.20 time: 0.029906 Sep 18 07:02:40.752073 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1138/2852] 10.222.0.20 () {60 vars in 1280 bytes} [Sun Sep 18 07:02:40 2022] DELETE /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 07:02:40.756592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:40.757204 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 2ced1b99-214c-4d5f-957c-e92b0bda948a] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:40.763138 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:40.763628 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:40.764065 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:40.764759 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:40.765209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:40.765632 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:40.779427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] HTTP exception thrown: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Request method failure captured: Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a HTTP/1.1 Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrn1lB3_UfTsYaV5VDBK-ypfzlj8urkIAwTPw5Yd2WBNRG2q1UHomYg5YU0Q_Gl1eVXe7O3KzhOZQBsmLKDSwprGT0ibDlrMshfvA4R7vtAqET5IlKrB0SXuHk2TN8wUSFWEaynYf04KL8MaFpR88nkIfYDjj1PbzrH7W__S3Z_5VEc Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:02:40.781073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 00399347c76d451d94d45f4edc3cf416 Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.783619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '2ced1b99-214c-4d5f-957c-e92b0bda948a'} Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:02:40.784616 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. Sep 18 07:02:40.786225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:40.787777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Returning 404 to user: Instance 2ced1b99-214c-4d5f-957c-e92b0bda948a could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:02:40.787777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-34aa4e75-24f0-4f53-922b-0413e0588bfe tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a" status: 404 len: 111 microversion: 2.20 time: 0.030686 Sep 18 07:02:40.787777 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1715/2853] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:02:40 2022] GET /compute/v2.1/servers/2ced1b99-214c-4d5f-957c-e92b0bda948a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 18 07:02:41.918604 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:41.919107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:41.925450 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:41.926501 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:41.926974 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:41.927655 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:41.928109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:41.928531 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:41.941510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] HTTP exception thrown: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Request method failure captured: Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 HTTP/1.1 Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrn1lB3_UfTsYaV5VDBK-ypfzlj8urkIAwTPw5Yd2WBNRG2q1UHomYg5YU0Q_Gl1eVXe7O3KzhOZQBsmLKDSwprGT0ibDlrMshfvA4R7vtAqET5IlKrB0SXuHk2TN8wUSFWEaynYf04KL8MaFpR88nkIfYDjj1PbzrH7W__S3Z_5VEc Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 07:02:41.943691 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 00399347c76d451d94d45f4edc3cf416 Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.945510 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '5ed3cfa3-6093-4c26-9877-01470acd2e16'} Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:02:41.946536 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.948259 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.949893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Returning 404 to user: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:02:41.949893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-39ec2dc3-e295-4e8e-b77b-0be38e616ac1 tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "DELETE /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 404 len: 111 microversion: 2.20 time: 0.030571 Sep 18 07:02:41.949893 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1139/2854] 10.222.0.20 () {60 vars in 1280 bytes} [Sun Sep 18 07:02:41 2022] DELETE /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 07:02:41.953509 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:41.954118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] [instance: 5ed3cfa3-6093-4c26-9877-01470acd2e16] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:41.960526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:41.961007 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:41.961441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:41.962122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:41.962584 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:41.963030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:41.976224 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] HTTP exception thrown: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Request method failure captured: Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 HTTP/1.1 Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrn1lB3_UfTsYaV5VDBK-ypfzlj8urkIAwTPw5Yd2WBNRG2q1UHomYg5YU0Q_Gl1eVXe7O3KzhOZQBsmLKDSwprGT0ibDlrMshfvA4R7vtAqET5IlKrB0SXuHk2TN8wUSFWEaynYf04KL8MaFpR88nkIfYDjj1PbzrH7W__S3Z_5VEc Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:02:41.977756 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/6fbf7eeb158e4a72bd8133faa1366e0d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 6fbf7eeb158e4a72bd8133faa1366e0d Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1264805537 Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 00399347c76d451d94d45f4edc3cf416 Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1264805537-project Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.979594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '5ed3cfa3-6093-4c26-9877-01470acd2e16'} Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:02:41.980470 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. Sep 18 07:02:41.981852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:41.983442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] Returning 404 to user: Instance 5ed3cfa3-6093-4c26-9877-01470acd2e16 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:02:41.983442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-321db828-11ac-46a9-9fd4-b0d6b7a9845e tempest-AttachVolumeShelveTestJSON-1264805537 tempest-AttachVolumeShelveTestJSON-1264805537-project] 10.222.0.20 "GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16" status: 404 len: 111 microversion: 2.20 time: 0.029480 Sep 18 07:02:41.983442 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1716/2855] 10.222.0.20 () {60 vars in 1277 bytes} [Sun Sep 18 07:02:41 2022] GET /compute/v2.1/servers/5ed3cfa3-6093-4c26-9877-01470acd2e16 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 07:02:43.476255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:43.476880 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:43.483653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:43.484249 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:43.484803 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:43.485588 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:43.486135 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:43.486677 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:43.515861 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Going to try to terminate instance {{(pid=120380) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 07:02:43.618423 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-095a7995-3c80-47e2-8b8f-18cd43bf4af7 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "DELETE /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 204 len: 0 microversion: 2.1 time: 0.144525 Sep 18 07:02:43.623200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1140/2856] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 07:02:43 2022] DELETE /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 07:02:43.628758 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:43.630049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:43.636863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:43.637587 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:43.638237 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:43.639209 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:43.639840 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:43.640619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:43.784318 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:43.793733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b3ac11de-a5fe-43af-b699-a2c93e45749a tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1576 microversion: 2.1 time: 0.169541 Sep 18 07:02:43.794317 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1717/2857] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:43 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1576 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:02:44.819111 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:44.819819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:44.831351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:44.832276 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:44.832887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:44.833789 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:44.834523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:44.835952 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:45.070217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:45.080761 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d9f953d5-a898-440e-a6c8-e7c3343e29c8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1576 microversion: 2.1 time: 0.266708 Sep 18 07:02:45.095948 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1141/2858] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:44 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1576 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:46.036096 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-e0bd490a-de0d-4243-824d-513cb7f81e8b req-0e7ca224-98b9-4b29-9e8e-52b74da83b83 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5cdcc319-ec7c-4d46-a87f-996f83c03821", "name": "network-vif-deleted", "tag": "7ea7f43d-b1b2-439b-a869-982bacaa30c8"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:02:46.047576 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-e0bd490a-de0d-4243-824d-513cb7f81e8b req-0e7ca224-98b9-4b29-9e8e-52b74da83b83 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:46.048049 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-e0bd490a-de0d-4243-824d-513cb7f81e8b req-0e7ca224-98b9-4b29-9e8e-52b74da83b83 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:46.049299 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-e0bd490a-de0d-4243-824d-513cb7f81e8b req-0e7ca224-98b9-4b29-9e8e-52b74da83b83 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:46.062930 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-e0bd490a-de0d-4243-824d-513cb7f81e8b req-0e7ca224-98b9-4b29-9e8e-52b74da83b83 service nova] Creating event network-vif-deleted:7ea7f43d-b1b2-439b-a869-982bacaa30c8 for instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 on hv-cinder-85535 Sep 18 07:02:46.069095 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-e0bd490a-de0d-4243-824d-513cb7f81e8b req-0e7ca224-98b9-4b29-9e8e-52b74da83b83 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036668 Sep 18 07:02:46.073178 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1718/2859] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:02:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:02:46.108442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:46.109688 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:46.118964 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:46.119460 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:46.119901 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:46.120607 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:46.122289 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:46.124936 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:46.203897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:46.215122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5d46b984-2a56-4654-a213-2905bba0b458 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 200 len: 1224 microversion: 2.1 time: 0.111201 Sep 18 07:02:46.219520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1142/2860] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:46 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 1224 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:46.471888 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-764c8cd5-1598-46a3-a0cd-da3a3886bd14 tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Action: 'create', calling method: .version_select at 0x7fe671886f70>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1513561669", "metadata": {"Type": "work"}}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:02:46.797179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-764c8cd5-1598-46a3-a0cd-da3a3886bd14 tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.477500 Sep 18 07:02:46.798429 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1719/2861] 10.222.0.20 () {60 vars in 1156 bytes} [Sun Sep 18 07:02:46 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 480 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 18 07:02:46.813981 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0fd05b54-3eb0-45b4-af8c-22bc15a3ed4f tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Calling method '.version_select at 0x7fe673d39f70>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:46.880181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0fd05b54-3eb0-45b4-af8c-22bc15a3ed4f tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf" status: 200 len: 353 microversion: 2.1 time: 0.070087 Sep 18 07:02:46.880839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1143/2862] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 07:02:46 2022] GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf => generated 353 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:02:47.243525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:47.244252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:47.251591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:47.252020 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:47.252461 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:47.253143 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:47.253596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:47.254659 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:47.267579 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] HTTP exception thrown: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Request method failure captured: Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 HTTP/1.1 Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJsHwP6HAV62fkLOiQ3vPuoYHfp1Z9xkdqSmTuwfMY_LHM9wYfsLVFiPbBq7p9R7RfpiZ5fmqUkmEHIg40p79sRdnZjKKH9eyr6xj9C_mgNAVmfdnPagvZ8Rcy4eTVfBiMeAcxbzHLVFgirfm03gJXVA-RewKu78_2UKeO2pW8Mpf18A Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:02:47.269170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestVolumeBootPattern-548700184 Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: b997560479c642e0ac43c0b08a1bf157 Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:02:47.271006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '5cdcc319-ec7c-4d46-a87f-996f83c03821'} Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:02:47.272099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:02:47.273746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:47.275303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Returning 404 to user: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:02:47.275303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fc2eb025-b94d-4250-9531-a7df9a2e49cf tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 404 len: 111 microversion: 2.1 time: 0.031418 Sep 18 07:02:47.275303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1720/2863] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:47 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:02:47.364090 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-368809729", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-671757144", "security_groups": [{"name": "tempest-secgroup-smoke-1568521079"}], "block_device_mapping_v2": [{"uuid": "18f4a776-d73f-4c69-a733-33b5145cc9bc", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c9fc9740-ace6-449a-b59c-d98e771aa3a8"}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:02:47.486035 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] validate_networks() for [('c9fc9740-ace6-449a-b59c-d98e771aa3a8', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 07:02:47.885910 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 07:02:47.886336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 07:02:47.886846 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 07:02:47.887298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 07:02:47.887709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 07:02:47.894068 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1778ac0b-2737-4076-bcc8-e9b21714152c tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Calling method '.version_select at 0x7fe6701a71f0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:47.896999 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:47.897492 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:47.897915 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:47.898670 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:47.899134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:47.899558 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:47.916502 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Getting quotas for project f707372e1ec948dcb7e89b9a046cccfc. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 07:02:47.920495 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Getting quotas for user b997560479c642e0ac43c0b08a1bf157 and project f707372e1ec948dcb7e89b9a046cccfc. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 07:02:47.941799 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1778ac0b-2737-4076-bcc8-e9b21714152c tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf" status: 200 len: 354 microversion: 2.1 time: 0.051611 Sep 18 07:02:47.942598 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1721/2864] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 07:02:47 2022] GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf => generated 354 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:02:47.955064 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d30b9697-80e3-4dd6-9a4f-3c98f8f19859 tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Calling method '.version_select at 0x7fe67051a0d0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:47.978262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 07:02:47.980430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='18f4a776-d73f-4c69-a733-33b5145cc9bc',volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 07:02:48.002745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d30b9697-80e3-4dd6-9a4f-3c98f8f19859 tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf" status: 200 len: 354 microversion: 2.1 time: 0.051276 Sep 18 07:02:48.003445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1722/2865] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 07:02:47 2022] GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf => generated 354 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:02:48.021821 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b2cdcae8-721d-44e9-b535-f6b435d06557 tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Calling method '.version_select at 0x7fe6701bb310>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:48.087618 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b2cdcae8-721d-44e9-b535-f6b435d06557 tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf" status: 202 len: 0 microversion: 2.1 time: 0.070261 Sep 18 07:02:48.088483 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1723/2866] 10.222.0.20 () {58 vars in 1248 bytes} [Sun Sep 18 07:02:48 2022] DELETE /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf => generated 0 bytes in 72 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 07:02:48.097260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-687ec354-b28b-4506-9a35-0415f92d68ab tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Calling method '.version_select at 0x7fe67051a940>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:48.121156 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a4b4bdc4-238d-42f5-a27f-671b5e7c256b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.761844 Sep 18 07:02:48.122132 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1144/2867] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 07:02:47 2022] POST /compute/v2.1/servers => generated 406 bytes in 763 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 07:02:48.133586 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-781cfe86-a0d7-4cd5-abc0-c7739e8396a5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:48.134349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-781cfe86-a0d7-4cd5-abc0-c7739e8396a5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:48.154754 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-687ec354-b28b-4506-9a35-0415f92d68ab tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf" status: 200 len: 353 microversion: 2.1 time: 0.061640 Sep 18 07:02:48.155434 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1724/2868] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 07:02:48 2022] GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf => generated 353 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:02:48.189409 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-781cfe86-a0d7-4cd5-abc0-c7739e8396a5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:48.196420 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-781cfe86-a0d7-4cd5-abc0-c7739e8396a5 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1093 microversion: 2.1 time: 0.066773 Sep 18 07:02:48.196971 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1145/2869] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:48 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1093 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:49.169593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70d35c07-d9ce-4336-ad6d-103514eb10ed tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Calling method '.version_select at 0x7fe6704c84c0>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:49.199345 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-70d35c07-d9ce-4336-ad6d-103514eb10ed tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] HTTP exception thrown: Volume 3db5a6f3-a41c-4d92-b0de-3f713125a2cf could not be found. Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70d35c07-d9ce-4336-ad6d-103514eb10ed tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Request method failure captured: Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf HTTP/1.1 Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJsKWZKsv2Hp50yWdod7jLbGD5G5TQ4Z8d4eLHluyPWOlga8Lc33X37aVZGIiKrbmE1kRgYrtljwgXhG8XRkOdCW3ajPCnsi85vYJ9qKw1YuJk6UrxtTc_aLCsQCSDr722Xzz7sOh-l52gx8ZXBGB9qlyIm1cmRBIg30Udw1TxJ6kmLc Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 2b163e3d06f44f438429d4c4d7938efc Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesGetTestJSON-471956436 Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:02:49.201566 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2b163e3d06f44f438429d4c4d7938efc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_2b163e3d06f44f438429d4c4d7938efc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/2b163e3d06f44f438429d4c4d7938efc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/2b163e3d06f44f438429d4c4d7938efc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesGetTestJSON-471956436 Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 2b163e3d06f44f438429d4c4d7938efc Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesGetTestJSON-471956436 Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesGetTestJSON-471956436-project Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 519d638b49f54728892c31be780d062e Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesGetTestJSON-471956436-project Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Volume 3db5a6f3-a41c-4d92-b0de-3f713125a2cf could not be found. Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '3db5a6f3-a41c-4d92-b0de-3f713125a2cf'} Sep 18 07:02:49.203334 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 3db5a6f3-a41c-4d92-b0de-3f713125a2cf could not be found. (HTTP 404) (Request-ID: req-dea19cc2-a25c-479e-9cba-22d6d0fbf646) Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:49.204491 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 07:02:49.206103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 3db5a6f3-a41c-4d92-b0de-3f713125a2cf could not be found. Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 18 07:02:49.207880 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 3db5a6f3-a41c-4d92-b0de-3f713125a2cf could not be found. Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 3db5a6f3-a41c-4d92-b0de-3f713125a2cf could not be found. Sep 18 07:02:49.209482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:02:49.212053 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-70d35c07-d9ce-4336-ad6d-103514eb10ed tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] Returning 404 to user: Volume 3db5a6f3-a41c-4d92-b0de-3f713125a2cf could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:02:49.212502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-70d35c07-d9ce-4336-ad6d-103514eb10ed tempest-VolumesGetTestJSON-471956436 tempest-VolumesGetTestJSON-471956436-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf" status: 404 len: 109 microversion: 2.1 time: 0.046766 Sep 18 07:02:49.213529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1725/2870] 10.222.0.20 () {58 vars in 1245 bytes} [Sun Sep 18 07:02:49 2022] GET /compute/v2.1/os-volumes/3db5a6f3-a41c-4d92-b0de-3f713125a2cf => generated 109 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:02:49.224301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:49.226011 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:49.232934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:49.233439 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:49.233887 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:49.234606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:49.235123 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:49.235552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:49.577338 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:49.589368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-76919477-3657-48a8-a712-d017b34f2f87 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1209 microversion: 2.1 time: 0.370565 Sep 18 07:02:49.590793 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1146/2871] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:49 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1209 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:02:50.619372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:50.620510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:50.629762 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:50.630679 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:50.631288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:50.632486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:50.633365 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:50.634127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:50.802763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:50.814851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c2e85fdd-c72c-4370-824a-f7b37cd18633 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.200800 Sep 18 07:02:50.815708 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1726/2872] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:50 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:50.979547 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-3cc28f45-08ea-4d36-9619-b4a9924ed0b0 req-5afec812-5e0d-4d7c-8e6e-d33e213a730d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4726cd7a-c664-4498-a510-2fffeb964af3", "tag": "3d68cbf6-c73d-4226-848b-9c00e71a04b4"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:02:50.993498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-3cc28f45-08ea-4d36-9619-b4a9924ed0b0 req-5afec812-5e0d-4d7c-8e6e-d33e213a730d service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:50.993980 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-3cc28f45-08ea-4d36-9619-b4a9924ed0b0 req-5afec812-5e0d-4d7c-8e6e-d33e213a730d service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:50.995000 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-3cc28f45-08ea-4d36-9619-b4a9924ed0b0 req-5afec812-5e0d-4d7c-8e6e-d33e213a730d service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:51.008836 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-3cc28f45-08ea-4d36-9619-b4a9924ed0b0 req-5afec812-5e0d-4d7c-8e6e-d33e213a730d service nova] Creating event network-changed:3d68cbf6-c73d-4226-848b-9c00e71a04b4 for instance 4726cd7a-c664-4498-a510-2fffeb964af3 on hv-cinder-85535 Sep 18 07:02:51.015366 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-3cc28f45-08ea-4d36-9619-b4a9924ed0b0 req-5afec812-5e0d-4d7c-8e6e-d33e213a730d service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040237 Sep 18 07:02:51.016540 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1147/2873] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:02:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:02:51.838804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:51.839464 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:51.845706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:51.846197 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:51.846666 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:51.847418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:51.847962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:51.848402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:51.994003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:52.005232 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1426d260-4812-4392-9346-257a5de65881 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.170553 Sep 18 07:02:52.005907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1727/2874] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:51 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:53.028445 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:53.029200 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:53.036203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:53.036697 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:53.037126 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:53.037818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:53.038272 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:53.038739 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:53.217541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:53.225894 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-287b3fe5-011b-422d-ba60-5cbcf527ee88 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.201627 Sep 18 07:02:53.226472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1148/2875] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:53 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:54.249113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:54.249723 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:54.257022 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:54.257503 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:54.257937 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:54.258652 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:54.259101 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:54.259524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:54.403261 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:54.411417 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d3ebc0d3-5d25-4b9d-8c3e-16c53fbb5b9f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.166375 Sep 18 07:02:54.411984 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1728/2876] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:54 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:55.435704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:55.436302 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:55.443368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:55.443850 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:55.444288 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:55.444991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:55.445443 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:55.445864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:55.583797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:55.591425 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f7c0480e-53ee-43eb-bd65-f6bd4c44ea1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.159643 Sep 18 07:02:55.591977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1149/2877] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:55 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:02:56.613234 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:56.613852 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:56.620500 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:56.620981 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:56.621412 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:56.622092 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:56.622820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:56.623246 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:56.756643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:56.764323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-b40647aa-b89b-4a3a-9d9c-5995602ecc19 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.154885 Sep 18 07:02:56.764883 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1729/2878] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:56 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:57.789689 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:57.790395 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:57.797246 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:57.797870 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:57.798438 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:57.799222 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:57.799801 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:57.805979 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:57.907059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:57.940229 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:57.948368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c9f480ac-d05b-4162-9135-7053feeaa94c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.163462 Sep 18 07:02:57.948929 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1150/2879] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:57 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:02:58.971923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:02:58.972751 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:02:58.979869 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:58.980416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:58.980916 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:58.981667 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:02:58.982185 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:02:58.982733 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:02:59.112435 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:02:59.120329 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-fe7f7506-0460-4d20-b41b-9124f13dbbf8 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.152201 Sep 18 07:02:59.120975 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1730/2880] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:02:58 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:03:00.143026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:03:00.143635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:03:00.145556 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=8 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:00.151715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:00.152193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:00.152625 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:00.153304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:00.153747 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:00.154164 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:00.297580 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:03:00.305960 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a85eee9e-cc56-4691-b0ff-236b4cdae718 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.166674 Sep 18 07:03:00.306466 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1151/2881] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:03:00 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:03:01.327688 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:03:01.328517 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:03:01.335520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:01.336003 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:01.336448 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:01.337131 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:01.337585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:01.338011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:01.468624 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:03:01.476294 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-1f745649-6be6-4e42-8868-81eed60cc1f6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.152437 Sep 18 07:03:01.476865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1731/2882] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:03:01 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:03:01.958051 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-89d4dd9a-1b35-4a97-8bf5-374559024493 req-a410b8e8-d09b-41dd-a1c0-d957045b54ed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4726cd7a-c664-4498-a510-2fffeb964af3", "name": "network-vif-plugged", "status": "completed", "tag": "3d68cbf6-c73d-4226-848b-9c00e71a04b4"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:03:01.968326 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-89d4dd9a-1b35-4a97-8bf5-374559024493 req-a410b8e8-d09b-41dd-a1c0-d957045b54ed service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:01.968811 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-89d4dd9a-1b35-4a97-8bf5-374559024493 req-a410b8e8-d09b-41dd-a1c0-d957045b54ed service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:01.969245 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-89d4dd9a-1b35-4a97-8bf5-374559024493 req-a410b8e8-d09b-41dd-a1c0-d957045b54ed service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:01.980848 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-89d4dd9a-1b35-4a97-8bf5-374559024493 req-a410b8e8-d09b-41dd-a1c0-d957045b54ed service nova] Creating event network-vif-plugged:3d68cbf6-c73d-4226-848b-9c00e71a04b4 for instance 4726cd7a-c664-4498-a510-2fffeb964af3 on hv-cinder-85535 Sep 18 07:03:01.985868 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-89d4dd9a-1b35-4a97-8bf5-374559024493 req-a410b8e8-d09b-41dd-a1c0-d957045b54ed service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031489 Sep 18 07:03:01.986522 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1152/2883] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:03:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:03:02.499361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:03:02.499968 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:03:02.506972 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:02.507453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:02.507889 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:02.508586 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:02.509061 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:02.509486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:02.635627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:03:02.643292 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-682c8818-4ddc-4f3c-8c44-f815b7155603 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1265 microversion: 2.1 time: 0.148106 Sep 18 07:03:02.643854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1732/2884] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:03:02 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1265 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:03:03.668134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:03:03.668752 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:03:03.674898 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:03.675375 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:03.675807 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:03.676483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:03.676931 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:03.677351 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:03.806797 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:03:03.814633 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5227202a-7255-4130-b679-dda638fc631c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1453 microversion: 2.1 time: 0.151324 Sep 18 07:03:03.815202 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1153/2885] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:03:03 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1453 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:03:03.836945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:03:03.837675 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:03:03.844127 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:03.844612 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:03.845056 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:03.845746 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:03.846205 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:03.846672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:03.985122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:03:03.992956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-ca357a59-e704-4897-a27c-9a4c0e5fd9e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1453 microversion: 2.1 time: 0.160349 Sep 18 07:03:03.993542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1733/2886] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:03:03 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1453 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:03:05.976146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-ae080dff-b4cb-4157-aa69-7a5ae81bc900 req-0f059441-87c1-423a-a6d9-7f3b6118cc4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4726cd7a-c664-4498-a510-2fffeb964af3", "tag": "3d68cbf6-c73d-4226-848b-9c00e71a04b4"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:03:05.981437 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=59,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:05.987144 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ae080dff-b4cb-4157-aa69-7a5ae81bc900 req-0f059441-87c1-423a-a6d9-7f3b6118cc4e service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:03:05.987614 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ae080dff-b4cb-4157-aa69-7a5ae81bc900 req-0f059441-87c1-423a-a6d9-7f3b6118cc4e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:03:05.988206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ae080dff-b4cb-4157-aa69-7a5ae81bc900 req-0f059441-87c1-423a-a6d9-7f3b6118cc4e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:03:05.993165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:06.001134 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-ae080dff-b4cb-4157-aa69-7a5ae81bc900 req-0f059441-87c1-423a-a6d9-7f3b6118cc4e service nova] Creating event network-changed:3d68cbf6-c73d-4226-848b-9c00e71a04b4 for instance 4726cd7a-c664-4498-a510-2fffeb964af3 on hv-cinder-85535 Sep 18 07:03:06.007401 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-ae080dff-b4cb-4157-aa69-7a5ae81bc900 req-0f059441-87c1-423a-a6d9-7f3b6118cc4e service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035426 Sep 18 07:03:06.011086 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1154/2887] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:03:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:04:42.144405 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:42.145169 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:42.146222 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 07:04:42.146369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Sun Sep 18 07:04:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.20) !!! Sep 18 07:04:42.151829 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-833785952", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-671757144", "security_groups": [{"name": "tempest-secgroup-smoke-1568521079"}], "block_device_mapping_v2": [{"uuid": "dd25c1dc-fb2c-4885-9616-2add85ad488d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "c9fc9740-ace6-449a-b59c-d98e771aa3a8"}]}} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:04:42.264917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.network.neutron [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] validate_networks() for [('c9fc9740-ace6-449a-b59c-d98e771aa3a8', None, None, None, None, None)] {{(pid=120381) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 07:04:42.715516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Flavor limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 07:04:42.715944 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Image limits 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 07:04:42.716420 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Flavor pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 07:04:42.716832 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Image pref 0:0:0 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 07:04:42.717249 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.virt.hardware [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120381) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 07:04:42.726495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:42.727004 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:42.727440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:42.728165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:42.728617 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:42.729041 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:42.745561 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Getting quotas for project f707372e1ec948dcb7e89b9a046cccfc. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 07:04:42.749485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.quota [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Getting quotas for user b997560479c642e0ac43c0b08a1bf157 and project f707372e1ec948dcb7e89b9a046cccfc. Resources: {'cores', 'instances', 'ram'} {{(pid=120381) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 07:04:42.804281 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Going to run 1 instances... {{(pid=120381) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 07:04:42.806397 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='dd25c1dc-fb2c-4885-9616-2add85ad488d',volume_size=None,volume_type=None)] {{(pid=120381) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 07:04:42.937392 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c35d2f42-2899-4b24-b60c-442c3d3cf6e1 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.794604 Sep 18 07:04:42.938032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1734/2888] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 07:04:42 2022] POST /compute/v2.1/servers => generated 406 bytes in 796 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 07:04:42.949350 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 07:04:42.949872 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Sun Sep 18 07:04:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db (ip 10.222.0.20) !!! Sep 18 07:04:42.957503 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-42cd5b7d-c655-4f38-a250-053ab35796fc tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:42.958380 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-42cd5b7d-c655-4f38-a250-053ab35796fc tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:43.014967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-42cd5b7d-c655-4f38-a250-053ab35796fc tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:43.021851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-42cd5b7d-c655-4f38-a250-053ab35796fc tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1093 microversion: 2.1 time: 0.075878 Sep 18 07:04:43.022975 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1155/2889] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:42 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1093 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:44.045165 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:44.045778 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:44.052838 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:44.053323 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:44.053761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:44.054496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:44.055129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:44.055557 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:44.126539 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:44.135449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a4d0e00b-7ed3-477b-ab93-823a194eef20 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1209 microversion: 2.1 time: 0.094335 Sep 18 07:04:44.136225 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1735/2890] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:44 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1209 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:45.159674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:45.160217 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:45.168363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:45.168864 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:45.169307 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:45.170017 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:45.170471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:45.170932 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:45.531119 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:45.539610 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-001f1c54-a929-4da8-bbfe-df7f2d491330 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.383998 Sep 18 07:04:45.540176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1156/2891] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:45 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:45.800144 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-d82dc1e1-ce7d-4b5c-b76b-34b049e1db49 req-a146ea12-6a44-43d5-9acf-76578d7a47b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1f648ad9-e7ac-40a4-9901-27d437beb2db", "tag": "e2178ad9-ae34-4f40-ac7c-93441f8a5887"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:04:45.809516 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-d82dc1e1-ce7d-4b5c-b76b-34b049e1db49 req-a146ea12-6a44-43d5-9acf-76578d7a47b9 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:45.809997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-d82dc1e1-ce7d-4b5c-b76b-34b049e1db49 req-a146ea12-6a44-43d5-9acf-76578d7a47b9 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:45.810523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-d82dc1e1-ce7d-4b5c-b76b-34b049e1db49 req-a146ea12-6a44-43d5-9acf-76578d7a47b9 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:45.821906 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-d82dc1e1-ce7d-4b5c-b76b-34b049e1db49 req-a146ea12-6a44-43d5-9acf-76578d7a47b9 service nova] Creating event network-changed:e2178ad9-ae34-4f40-ac7c-93441f8a5887 for instance 1f648ad9-e7ac-40a4-9901-27d437beb2db on hv-cinder-85535 Sep 18 07:04:45.827129 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-d82dc1e1-ce7d-4b5c-b76b-34b049e1db49 req-a146ea12-6a44-43d5-9acf-76578d7a47b9 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.152489 Sep 18 07:04:45.828077 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1736/2892] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:04:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:04:46.562675 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:46.563421 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:46.569919 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:46.570403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:46.570853 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:46.571541 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:46.571992 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:46.572413 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:46.774714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:46.782715 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c04e2782-2e85-4982-96a0-e64bbf2484ab tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.223860 Sep 18 07:04:46.783293 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1157/2893] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:46 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:47.806275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:47.806992 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:47.813538 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:47.814015 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:47.814447 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:47.815147 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:47.815596 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:47.816018 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:47.943820 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:47.952047 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d6d196cc-3adc-44f8-bdea-3952fc636778 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.150494 Sep 18 07:04:47.952602 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1737/2894] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:47 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:48.974552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:48.975193 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:48.981098 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:48.981584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:48.982026 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:48.982745 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:48.983203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:48.983628 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:49.116353 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:49.124221 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-134cc8de-fcef-468e-88ab-5ba547c6eb58 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.153741 Sep 18 07:04:49.124787 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1158/2895] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:48 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:50.148159 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:50.148770 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:50.154776 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:50.155260 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:50.155704 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:50.156381 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:50.156833 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:50.157254 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:50.284288 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:50.291945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-09a934ff-4f37-436e-a98b-cea573b733cd tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.148366 Sep 18 07:04:50.292505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1738/2896] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:50 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:51.313858 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:51.314468 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:51.320921 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:51.321403 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:51.321856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:51.322521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:51.323296 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:51.324925 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:51.454336 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:51.462668 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a94e6740-8d2b-46fb-bf3a-3816299d7605 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.152600 Sep 18 07:04:51.463231 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1159/2897] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:51 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:52.494527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:52.495432 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:52.501995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:52.502507 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:52.503002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:52.503721 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:52.504170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:52.504591 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:52.638718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:52.646320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-835a1da6-71a6-473f-a97f-0930776efc1f tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.156283 Sep 18 07:04:52.646915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1739/2898] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:52 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:53.669195 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:53.669812 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:53.672178 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:53.677619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:53.678106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:53.678544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:53.679771 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:53.680225 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:53.680652 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:53.811952 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:53.820431 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c574dc28-c43f-45ec-815f-b466f2dbd55d tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.154384 Sep 18 07:04:53.820986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1160/2899] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:53 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:54.842917 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:54.843521 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:54.845423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:54.846075 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=6 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:54.852614 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:54.853090 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:54.853519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:54.854198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:54.854678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:54.855102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:54.993033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:55.000874 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d03c4e1e-5ef5-4cff-ac49-31f6b2411b03 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.161725 Sep 18 07:04:55.001451 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1740/2900] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:54 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:04:56.024345 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:56.024958 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:56.031957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:56.032444 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:56.032913 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:56.033796 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:56.034397 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:56.034993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:56.165404 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:56.172977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-07698cbf-524e-49ef-bf09-c90f9b2fb194 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.152982 Sep 18 07:04:56.173539 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1161/2901] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:56 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:56.533117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-1bd5f12a-8eee-4ea0-9b59-987208ad5c0f req-a311d641-2ce8-494a-8d11-c79fe1fc74f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1f648ad9-e7ac-40a4-9901-27d437beb2db", "name": "network-vif-plugged", "status": "completed", "tag": "e2178ad9-ae34-4f40-ac7c-93441f8a5887"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:04:56.543564 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1bd5f12a-8eee-4ea0-9b59-987208ad5c0f req-a311d641-2ce8-494a-8d11-c79fe1fc74f7 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:56.544045 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1bd5f12a-8eee-4ea0-9b59-987208ad5c0f req-a311d641-2ce8-494a-8d11-c79fe1fc74f7 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:56.544482 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1bd5f12a-8eee-4ea0-9b59-987208ad5c0f req-a311d641-2ce8-494a-8d11-c79fe1fc74f7 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:56.557741 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-1bd5f12a-8eee-4ea0-9b59-987208ad5c0f req-a311d641-2ce8-494a-8d11-c79fe1fc74f7 service nova] Creating event network-vif-plugged:e2178ad9-ae34-4f40-ac7c-93441f8a5887 for instance 1f648ad9-e7ac-40a4-9901-27d437beb2db on hv-cinder-85535 Sep 18 07:04:56.562724 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-1bd5f12a-8eee-4ea0-9b59-987208ad5c0f req-a311d641-2ce8-494a-8d11-c79fe1fc74f7 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033262 Sep 18 07:04:56.563315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1741/2902] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:04:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:04:57.194682 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:57.195299 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:57.201681 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:57.202169 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:57.202635 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:57.203328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:57.203784 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:57.204211 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:57.329904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:57.338038 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a85ce120-a1ba-467b-a8e7-ce7111138428 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1265 microversion: 2.1 time: 0.147327 Sep 18 07:04:57.338619 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1162/2903] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:57 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1265 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:58.361546 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:58.362233 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:58.368809 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:58.369293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:58.369718 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:58.370672 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:58.371118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:58.371536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:58.505173 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:58.512945 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5ce5c8e5-fc34-4661-b2c3-d8e84ea3d913 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1453 microversion: 2.1 time: 0.156074 Sep 18 07:04:58.513515 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1742/2904] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:58 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1453 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:04:58.534724 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:04:58.535342 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:04:58.542139 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:58.542646 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:58.543087 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:58.543772 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:04:58.544228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:04:58.544657 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:04:58.676286 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:04:58.684535 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-050fccbb-9b3b-4df6-8067-af70372fafc2 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1453 microversion: 2.1 time: 0.153600 Sep 18 07:04:58.685096 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1163/2905] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:04:58 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1453 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:05:00.230033 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-1f6b2144-2584-4d50-bfe6-dbddf559282f req-a04e2256-7431-4422-bbf8-c044174867ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1f648ad9-e7ac-40a4-9901-27d437beb2db", "tag": "e2178ad9-ae34-4f40-ac7c-93441f8a5887"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:05:00.239940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1f6b2144-2584-4d50-bfe6-dbddf559282f req-a04e2256-7431-4422-bbf8-c044174867ad service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:05:00.240495 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1f6b2144-2584-4d50-bfe6-dbddf559282f req-a04e2256-7431-4422-bbf8-c044174867ad service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:05:00.241715 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-1f6b2144-2584-4d50-bfe6-dbddf559282f req-a04e2256-7431-4422-bbf8-c044174867ad service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:05:00.253559 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-1f6b2144-2584-4d50-bfe6-dbddf559282f req-a04e2256-7431-4422-bbf8-c044174867ad service nova] Creating event network-changed:e2178ad9-ae34-4f40-ac7c-93441f8a5887 for instance 1f648ad9-e7ac-40a4-9901-27d437beb2db on hv-cinder-85535 Sep 18 07:05:00.258630 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-1f6b2144-2584-4d50-bfe6-dbddf559282f req-a04e2256-7431-4422-bbf8-c044174867ad service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032851 Sep 18 07:05:00.259262 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1743/2906] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:05:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:05:21.277206 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:05:21.277700 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:05:21.279851 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=21 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:21.280497 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=49 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:21.286105 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:05:21.286601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:05:21.287062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:05:21.287744 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:05:21.288694 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:05:21.289146 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:05:21.304904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 HTTP/1.1 Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 07:05:21.306542 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.307823 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': 'cb5ac07d-bc0f-455d-a133-6abb2e30f695'} Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:05:21.308498 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.309608 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.310763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:05:21.310763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-2b4669bb-25d1-4dab-962c-4255fef8237e tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 404 len: 111 microversion: 2.1 time: 0.034631 Sep 18 07:05:21.310763 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1164/2907] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 07:05:21 2022] DELETE /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:05:21.314190 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:21.314955 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=44 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:21.318865 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:05:21.319604 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] [instance: cb5ac07d-bc0f-455d-a133-6abb2e30f695] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:05:21.327079 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:05:21.327594 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:05:21.328028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:05:21.328712 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:05:21.329164 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:05:21.329585 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:05:21.343270 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] HTTP exception thrown: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Request method failure captured: Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 HTTP/1.1 Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJrmq-FC55YWBXItZc4Vv4P_xH8GQICm6LLL77Dwtu4Ns1SE7UimvBf_Kp06ReWnoLeHLJB5GxJXpA57_X7u8DrrrB24n-71OVdb25YuoYRCizd4BG4NbDaZr5Y2XJZGb_beDLRW6nu1-txrTzx6p1ky0aE29fxC6UukbfgY7TDjKoYY Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:05:21.344864 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/0c5d2a3f2d944529b8c505fa23b5a10f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 0c5d2a3f2d944529b8c505fa23b5a10f Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1138226246 Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: d85160b01d824bc3a32b425fab98d2c7 Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-AttachVolumeNegativeTest-1138226246-project Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.346140 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'cb5ac07d-bc0f-455d-a133-6abb2e30f695'} Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:05:21.347011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. Sep 18 07:05:21.348293 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:05:21.349441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] Returning 404 to user: Instance cb5ac07d-bc0f-455d-a133-6abb2e30f695 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:05:21.349441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-74c48de3-d0d9-46ea-ac8d-f36b9c732a9b tempest-AttachVolumeNegativeTest-1138226246 tempest-AttachVolumeNegativeTest-1138226246-project] 10.222.0.20 "GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695" status: 404 len: 111 microversion: 2.1 time: 0.033617 Sep 18 07:05:21.349441 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1744/2908] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:05:21 2022] GET /compute/v2.1/servers/cb5ac07d-bc0f-455d-a133-6abb2e30f695 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:06:11.409969 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1161219226", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "networks": [{"uuid": "3b673732-0e1f-4638-951b-778a0f3d5cbf"}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:11.420731 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_api:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:11.421377 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:11.469360 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] validate_networks() for [('3b673732-0e1f-4638-951b-778a0f3d5cbf', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 07:06:11.812930 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 07:06:11.813378 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 07:06:11.813852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 07:06:11.814261 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 07:06:11.814711 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 07:06:11.824372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:11.824785 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:11.825211 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:11.825924 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:11.826368 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:11.826817 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:11.843145 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Getting quotas for project 523fb59392af43bcb2be01e17406cef7. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 07:06:11.848398 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Getting quotas for user 9842848655fe479bbea42fd0f2ca4ffa and project 523fb59392af43bcb2be01e17406cef7. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 07:06:11.860408 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 07:06:11.862748 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 07:06:11.907328 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-ebce85b1-4448-49cd-a523-b105776bdc9d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.502399 Sep 18 07:06:11.907973 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1165/2909] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 07:06:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 504 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 07:06:11.919416 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-43b79794-ef99-4452-86b1-630606d4fba6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:11.920030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-43b79794-ef99-4452-86b1-630606d4fba6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:11.922168 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:11.923122 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:11.977732 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-43b79794-ef99-4452-86b1-630606d4fba6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:11.984558 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-43b79794-ef99-4452-86b1-630606d4fba6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1214 microversion: 2.1 time: 0.069006 Sep 18 07:06:11.985153 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1745/2910] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:11 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1214 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:13.009792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:13.010478 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:13.017133 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:13.017613 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:13.018306 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:13.019149 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:13.020228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:13.020667 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:13.088137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:13.095749 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-24f2fe69-83d9-4797-a90c-a7fef2badbc3 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1284 microversion: 2.1 time: 0.091444 Sep 18 07:06:13.096301 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1166/2911] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:13 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1284 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:14.118783 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:14.119354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:14.127114 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:14.127601 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:14.128039 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:14.128727 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:14.129182 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:14.129606 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:14.359745 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:14.367641 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-a64ad0f8-e8f0-4701-b02f-9eeb81effb8d tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1314 microversion: 2.1 time: 0.253134 Sep 18 07:06:14.368402 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1746/2912] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:14 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1314 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:14.655886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-4ddcadba-1616-4b77-9512-09e33e38b8ed req-3d9162c7-aec3-4dee-a11b-ca110536441e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c326692a-db56-4b8e-83fc-68b31b4d9ef4", "tag": "347b75cd-2a47-4e71-b6ab-bb0ac193f519"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:14.665372 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-4ddcadba-1616-4b77-9512-09e33e38b8ed req-3d9162c7-aec3-4dee-a11b-ca110536441e service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:14.665845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-4ddcadba-1616-4b77-9512-09e33e38b8ed req-3d9162c7-aec3-4dee-a11b-ca110536441e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:14.666274 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-4ddcadba-1616-4b77-9512-09e33e38b8ed req-3d9162c7-aec3-4dee-a11b-ca110536441e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:14.677884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-4ddcadba-1616-4b77-9512-09e33e38b8ed req-3d9162c7-aec3-4dee-a11b-ca110536441e service nova] Creating event network-changed:347b75cd-2a47-4e71-b6ab-bb0ac193f519 for instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 on hv-cinder-85535 Sep 18 07:06:14.681833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR oslo.messaging._drivers.impl_rabbit [req-4ddcadba-1616-4b77-9512-09e33e38b8ed req-3d9162c7-aec3-4dee-a11b-ca110536441e service nova] [30a3a4b1-8749-43a6-8164-1025dc424aa6] AMQP server on 10.222.0.20:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 18 07:06:15.390286 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:15.390837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:15.397219 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:15.397694 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:15.398639 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:15.399315 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:15.399761 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:15.400180 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:15.526186 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:15.533784 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-942d24f5-f706-42f8-a775-7d3cf130832f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1314 microversion: 2.1 time: 0.147465 Sep 18 07:06:15.534373 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1747/2913] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:15 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1314 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:06:15.689996 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO oslo.messaging._drivers.impl_rabbit [req-4ddcadba-1616-4b77-9512-09e33e38b8ed req-3d9162c7-aec3-4dee-a11b-ca110536441e service nova] [30a3a4b1-8749-43a6-8164-1025dc424aa6] Reconnected to AMQP server on 10.222.0.20:5672 via [amqp] client with port 54864. Sep 18 07:06:15.693298 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-4ddcadba-1616-4b77-9512-09e33e38b8ed req-3d9162c7-aec3-4dee-a11b-ca110536441e service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 1.166907 Sep 18 07:06:15.693895 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1167/2914] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 1168 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:06:16.557583 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:16.558191 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:16.564610 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:16.565091 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:16.565525 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:16.566211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:16.566699 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:16.567125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:16.698592 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:16.709396 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-40120873-44bf-40a3-b31c-d6a8b9c422ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1314 microversion: 2.1 time: 0.156049 Sep 18 07:06:16.710275 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1748/2915] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:16 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:17.732892 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:17.733606 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:17.739704 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:17.740181 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:17.740609 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:17.741304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:17.742238 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:17.742686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:17.871089 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:17.878656 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1fb1a868-f6f8-4f9d-bc73-6c72791e1e3c tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1314 microversion: 2.1 time: 0.150616 Sep 18 07:06:17.879210 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1168/2916] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:17 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:18.902472 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:18.903256 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:18.909790 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:18.910282 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:18.910755 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:18.911459 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:18.911912 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:18.912337 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:19.047001 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:19.047432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-495dfbd2-b1ce-4b0b-906b-f9d4430676d3 req-6ac9b54a-ac03-4d85-b7c4-ba64a687c784 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c326692a-db56-4b8e-83fc-68b31b4d9ef4", "name": "network-vif-plugged", "status": "completed", "tag": "347b75cd-2a47-4e71-b6ab-bb0ac193f519"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:19.056284 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-84134775-67d1-41e5-b323-3e6e8fa6d673 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1314 microversion: 2.1 time: 0.158744 Sep 18 07:06:19.056861 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1749/2917] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:18 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:19.060630 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-495dfbd2-b1ce-4b0b-906b-f9d4430676d3 req-6ac9b54a-ac03-4d85-b7c4-ba64a687c784 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:19.061117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-495dfbd2-b1ce-4b0b-906b-f9d4430676d3 req-6ac9b54a-ac03-4d85-b7c4-ba64a687c784 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:19.061545 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-495dfbd2-b1ce-4b0b-906b-f9d4430676d3 req-6ac9b54a-ac03-4d85-b7c4-ba64a687c784 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:19.073331 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-495dfbd2-b1ce-4b0b-906b-f9d4430676d3 req-6ac9b54a-ac03-4d85-b7c4-ba64a687c784 service nova] Creating event network-vif-plugged:347b75cd-2a47-4e71-b6ab-bb0ac193f519 for instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 on hv-cinder-85535 Sep 18 07:06:19.078177 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-495dfbd2-b1ce-4b0b-906b-f9d4430676d3 req-6ac9b54a-ac03-4d85-b7c4-ba64a687c784 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100056 Sep 18 07:06:19.078777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1169/2918] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:06:20.080593 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:20.081235 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:20.088179 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:20.088661 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:20.089102 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:20.089787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:20.090240 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:20.090702 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:20.219198 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:20.226825 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-54609191-645a-47d9-a756-1fd91b21aeb9 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1314 microversion: 2.1 time: 0.150072 Sep 18 07:06:20.227391 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1750/2919] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:20 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:21.249709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:21.250406 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:21.256472 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:21.256949 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:21.257386 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:21.258062 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:21.258511 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:21.258954 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:21.388659 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:21.396262 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-eb126243-5d65-4fe7-821a-d1f26bcaca48 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1498 microversion: 2.1 time: 0.151208 Sep 18 07:06:21.396820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1170/2920] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:21 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1498 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:21.796940 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:21.797535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:21.804014 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:21.804510 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:21.804952 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:21.805645 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:21.806103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:21.806529 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:21.832305 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 07:06:21.907736 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-deae9e55-2877-41f8-b266-5d4afe1c6914 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "DELETE /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 204 len: 0 microversion: 2.1 time: 0.114182 Sep 18 07:06:21.908199 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1751/2921] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 07:06:21 2022] DELETE /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 07:06:21.916525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:21.917176 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:21.919107 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:21.919777 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:21.925589 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:21.926373 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:21.926943 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:21.927728 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:21.928186 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:21.928616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:22.053845 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:22.061985 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-3ab1e98e-446d-4a92-8f63-0e81095b5639 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1504 microversion: 2.1 time: 0.149342 Sep 18 07:06:22.062552 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1171/2922] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:21 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1504 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:23.091810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:23.092526 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:23.094486 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:23.101170 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:23.101651 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:23.102085 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:23.102804 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:23.103258 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:23.103682 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:23.263328 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:23.270938 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9512e974-3008-4ef3-a133-3239e6648dc6 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 200 len: 1504 microversion: 2.1 time: 0.183402 Sep 18 07:06:23.271512 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1752/2923] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:23 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 1504 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:23.673839 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-c82bdd8e-4eab-41a2-9baf-9056022766b7 req-6601c7ad-405e-4e36-b65a-ab58f19abb79 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c326692a-db56-4b8e-83fc-68b31b4d9ef4", "name": "network-vif-deleted", "tag": "347b75cd-2a47-4e71-b6ab-bb0ac193f519"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:23.685473 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-c82bdd8e-4eab-41a2-9baf-9056022766b7 req-6601c7ad-405e-4e36-b65a-ab58f19abb79 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:23.685957 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-c82bdd8e-4eab-41a2-9baf-9056022766b7 req-6601c7ad-405e-4e36-b65a-ab58f19abb79 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:23.686388 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-c82bdd8e-4eab-41a2-9baf-9056022766b7 req-6601c7ad-405e-4e36-b65a-ab58f19abb79 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:23.699751 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-c82bdd8e-4eab-41a2-9baf-9056022766b7 req-6601c7ad-405e-4e36-b65a-ab58f19abb79 service nova] Creating event network-vif-deleted:347b75cd-2a47-4e71-b6ab-bb0ac193f519 for instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 on hv-cinder-85535 Sep 18 07:06:23.704621 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-c82bdd8e-4eab-41a2-9baf-9056022766b7 req-6601c7ad-405e-4e36-b65a-ab58f19abb79 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034489 Sep 18 07:06:23.705208 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1172/2924] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:06:24.295073 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:24.295693 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: c326692a-db56-4b8e-83fc-68b31b4d9ef4] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:24.302027 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:24.302514 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:24.302987 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:24.303674 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:24.304128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:24.304555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:24.317485 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] HTTP exception thrown: Instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 could not be found. Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Request method failure captured: Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 HTTP/1.1 Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJsNhnklrO1ENM6wCDtESPdZQ4Ch9WcJ9N318z0zTaT-rxM4HbMt6UrkrFU_gnrfqBqSRud0WYeacdNSJPSHNwt-lQM1kF8nKHjH0thKMhHAAekWqhy5QPlnD62DySRxYS4vFMPYip0thECHFE-aH25sjdgcmELnoWKgtMtriYbFmfJ0 Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: 523fb59392af43bcb2be01e17406cef7 Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-VolumesActionsTest-759759597 Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:06:24.319132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/523fb59392af43bcb2be01e17406cef7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_523fb59392af43bcb2be01e17406cef7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/523fb59392af43bcb2be01e17406cef7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/523fb59392af43bcb2be01e17406cef7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-VolumesActionsTest-759759597 Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: 523fb59392af43bcb2be01e17406cef7 Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-VolumesActionsTest-759759597 Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-VolumesActionsTest-759759597-project Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: 9842848655fe479bbea42fd0f2ca4ffa Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-VolumesActionsTest-759759597-project Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 could not be found. Sep 18 07:06:24.320551 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': 'c326692a-db56-4b8e-83fc-68b31b4d9ef4'} Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 could not be found. Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:06:24.321428 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 could not be found. Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 could not be found. Sep 18 07:06:24.322958 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:24.323942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Returning 404 to user: Instance c326692a-db56-4b8e-83fc-68b31b4d9ef4 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:06:24.323942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-303b4f94-3fe6-4598-b38a-ada2df41ea17 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4" status: 404 len: 111 microversion: 2.1 time: 0.029552 Sep 18 07:06:24.323942 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1753/2925] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:24 2022] GET /compute/v2.1/servers/c326692a-db56-4b8e-83fc-68b31b4d9ef4 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:06:24.369995 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-435856366", "imageRef": "638cd16a-64c8-4e8c-b3dd-303357ab4082", "flavorRef": "452", "networks": [{"uuid": "3b673732-0e1f-4638-951b-778a0f3d5cbf"}]}} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:24.425896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.network.neutron [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] validate_networks() for [('3b673732-0e1f-4638-951b-778a0f3d5cbf', None, None, None, None, None)] {{(pid=120380) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 07:06:25.003315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Flavor limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 07:06:25.003875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Image limits 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 07:06:25.004242 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Flavor pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 07:06:25.004653 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Image pref 0:0:0 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 07:06:25.005059 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.virt.hardware [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120380) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 07:06:25.014162 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:25.014660 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:25.015093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:25.015808 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:25.016254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:25.016671 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:25.033014 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Getting quotas for project 523fb59392af43bcb2be01e17406cef7. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 07:06:25.036719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.quota [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Getting quotas for user 9842848655fe479bbea42fd0f2ca4ffa and project 523fb59392af43bcb2be01e17406cef7. Resources: {'cores', 'instances', 'ram'} {{(pid=120380) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 07:06:25.047194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Going to run 1 instances... {{(pid=120380) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 07:06:25.049530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='638cd16a-64c8-4e8c-b3dd-303357ab4082',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120380) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 07:06:25.095518 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-9dc6ca17-0a4b-4fae-80fb-0b31df67ba4f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.729988 Sep 18 07:06:25.096166 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1173/2926] 10.222.0.20 () {60 vars in 1147 bytes} [Sun Sep 18 07:06:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 731 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 07:06:25.106913 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c51049d5-5510-477d-b733-88a9bee3c045 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:25.107520 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c51049d5-5510-477d-b733-88a9bee3c045 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:25.161354 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c51049d5-5510-477d-b733-88a9bee3c045 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:25.168115 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c51049d5-5510-477d-b733-88a9bee3c045 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1213 microversion: 2.1 time: 0.064943 Sep 18 07:06:25.168995 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1754/2927] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:25 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1213 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:26.191453 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:26.192072 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:26.199886 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:26.200376 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:26.200818 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:26.201509 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:26.202117 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:26.202720 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:26.269101 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:26.277267 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-1003a078-e3c7-46e3-89d5-50b1321cd4d2 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1271 microversion: 2.1 time: 0.089829 Sep 18 07:06:26.277824 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1174/2928] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:26 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1271 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:27.299680 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:27.300368 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:27.307660 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:27.308230 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:27.308785 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:27.309471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:27.309921 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:27.310342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:27.466707 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-3e930b34-7fc7-4c3c-935b-4813589c99fc req-b69e30e8-bdd5-42e5-8439-6462b63bec0e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "623c2940-94eb-4476-81e7-6e0a20b6414b", "tag": "80aebc1c-91c6-4394-ab2d-9b8046186ebc"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:27.481937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-3e930b34-7fc7-4c3c-935b-4813589c99fc req-b69e30e8-bdd5-42e5-8439-6462b63bec0e service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:27.482865 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-3e930b34-7fc7-4c3c-935b-4813589c99fc req-b69e30e8-bdd5-42e5-8439-6462b63bec0e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:27.484137 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-3e930b34-7fc7-4c3c-935b-4813589c99fc req-b69e30e8-bdd5-42e5-8439-6462b63bec0e service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:27.502942 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-3e930b34-7fc7-4c3c-935b-4813589c99fc req-b69e30e8-bdd5-42e5-8439-6462b63bec0e service nova] Creating event network-changed:80aebc1c-91c6-4394-ab2d-9b8046186ebc for instance 623c2940-94eb-4476-81e7-6e0a20b6414b on hv-cinder-85535 Sep 18 07:06:27.510337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-3e930b34-7fc7-4c3c-935b-4813589c99fc req-b69e30e8-bdd5-42e5-8439-6462b63bec0e service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048964 Sep 18 07:06:27.511424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1175/2929] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:06:27.542238 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:27.558330 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-09edfef3-b48f-4c08-91fe-18e457e52442 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1313 microversion: 2.1 time: 0.262805 Sep 18 07:06:27.558967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1755/2930] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:27 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1313 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:06:28.318108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:28.318726 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:28.322200 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-6f009700-b9d4-44e0-bc27-4db62d06f982 req-06fe06ab-85ab-4eaf-9c2a-1ca33660eeb3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1f648ad9-e7ac-40a4-9901-27d437beb2db", "tag": "e2178ad9-ae34-4f40-ac7c-93441f8a5887"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:28.325601 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:28.326097 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:28.326537 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:28.327255 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:28.327888 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:28.328320 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:28.334923 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6f009700-b9d4-44e0-bc27-4db62d06f982 req-06fe06ab-85ab-4eaf-9c2a-1ca33660eeb3 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:28.335531 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6f009700-b9d4-44e0-bc27-4db62d06f982 req-06fe06ab-85ab-4eaf-9c2a-1ca33660eeb3 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:28.336103 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-6f009700-b9d4-44e0-bc27-4db62d06f982 req-06fe06ab-85ab-4eaf-9c2a-1ca33660eeb3 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:28.348837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-6f009700-b9d4-44e0-bc27-4db62d06f982 req-06fe06ab-85ab-4eaf-9c2a-1ca33660eeb3 service nova] Creating event network-changed:e2178ad9-ae34-4f40-ac7c-93441f8a5887 for instance 1f648ad9-e7ac-40a4-9901-27d437beb2db on hv-cinder-85535 Sep 18 07:06:28.354333 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Going to try to terminate instance {{(pid=120380) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 07:06:28.354837 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-6f009700-b9d4-44e0-bc27-4db62d06f982 req-06fe06ab-85ab-4eaf-9c2a-1ca33660eeb3 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037354 Sep 18 07:06:28.355763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1756/2931] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:06:28.431310 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-5cf82a83-c306-4d2c-aad2-bdca9d056eed tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 204 len: 0 microversion: 2.1 time: 0.117263 Sep 18 07:06:28.431897 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1176/2932] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 07:06:28 2022] DELETE /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 07:06:28.440607 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:28.441320 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:28.447524 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:28.448011 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:28.448453 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:28.449156 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:28.449619 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:28.450050 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:28.582457 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:28.583187 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:28.584048 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:28.590015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:28.590496 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:28.590986 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:28.591674 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:28.592122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:28.592709 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:28.593772 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-f5e3a875-d8a4-454a-9276-97e52dbd4698 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1578 microversion: 2.1 time: 0.157369 Sep 18 07:06:28.594763 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1757/2933] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:28 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1578 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:06:28.750791 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:28.758683 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-f1c0ec3b-33ca-4188-ba79-cefa04faad52 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1313 microversion: 2.1 time: 0.180435 Sep 18 07:06:28.759254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1177/2934] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:28 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1313 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:06:29.619522 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:29.620117 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:29.626657 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:29.626854 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:29.627297 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:29.627997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:29.628449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:29.628878 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:29.781015 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:29.782251 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:29.788699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:29.789442 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:29.790106 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:29.790977 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:29.791429 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:29.791852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:29.825750 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:29.834107 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-90dc315a-9820-41e6-b24b-49324fc310ba tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1578 microversion: 2.1 time: 0.220555 Sep 18 07:06:29.847627 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1758/2935] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:29 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1578 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:29.937327 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:29.944926 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-0535f1dd-9248-4fa0-951b-1401e93dc789 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1313 microversion: 2.1 time: 0.167655 Sep 18 07:06:29.945483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1178/2936] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:29 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1313 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:06:30.858032 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:30.859099 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:30.867683 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:30.868440 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:30.868907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:30.869615 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:30.870071 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:30.870535 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:30.896551 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-fe0877c6-4fa0-4ab6-bd9a-1c944b6048f9 req-8ab460f9-d62b-4c45-bdf5-d870b5b289c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1f648ad9-e7ac-40a4-9901-27d437beb2db", "name": "network-vif-deleted", "tag": "e2178ad9-ae34-4f40-ac7c-93441f8a5887"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:30.911616 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-fe0877c6-4fa0-4ab6-bd9a-1c944b6048f9 req-8ab460f9-d62b-4c45-bdf5-d870b5b289c1 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:30.912363 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-fe0877c6-4fa0-4ab6-bd9a-1c944b6048f9 req-8ab460f9-d62b-4c45-bdf5-d870b5b289c1 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:30.912937 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-fe0877c6-4fa0-4ab6-bd9a-1c944b6048f9 req-8ab460f9-d62b-4c45-bdf5-d870b5b289c1 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:30.927470 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-fe0877c6-4fa0-4ab6-bd9a-1c944b6048f9 req-8ab460f9-d62b-4c45-bdf5-d870b5b289c1 service nova] Creating event network-vif-deleted:e2178ad9-ae34-4f40-ac7c-93441f8a5887 for instance 1f648ad9-e7ac-40a4-9901-27d437beb2db on hv-cinder-85535 Sep 18 07:06:30.936203 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-fe0877c6-4fa0-4ab6-bd9a-1c944b6048f9 req-8ab460f9-d62b-4c45-bdf5-d870b5b289c1 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043305 Sep 18 07:06:30.937108 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1179/2937] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:06:30.968304 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:30.969093 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:30.975448 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:30.975851 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:30.976016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:30.976343 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:30.977065 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:30.977896 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:30.978002 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:30.984769 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9f569ec1-941f-405c-aae7-7f5eb442640c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 200 len: 1391 microversion: 2.1 time: 0.130603 Sep 18 07:06:30.985805 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1759/2938] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:30 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 1391 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:31.119743 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:31.127411 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-933add09-5b7e-4f75-aa4d-00d148444070 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1313 microversion: 2.1 time: 0.163622 Sep 18 07:06:31.127908 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1180/2939] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:30 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1313 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:32.008860 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:32.009471 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 1f648ad9-e7ac-40a4-9901-27d437beb2db] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:32.016035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:32.016542 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:32.017017 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:32.017706 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:32.018162 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:32.018603 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:32.032213 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] HTTP exception thrown: Instance 1f648ad9-e7ac-40a4-9901-27d437beb2db could not be found. Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Request method failure captured: Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db HTTP/1.1 Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJsHwP6HAV62fkLOiQ3vPuoYHfp1Z9xkdqSmTuwfMY_LHM9wYfsLVFiPbBq7p9R7RfpiZ5fmqUkmEHIg40p79sRdnZjKKH9eyr6xj9C_mgNAVmfdnPagvZ8Rcy4eTVfBiMeAcxbzHLVFgirfm03gJXVA-RewKu78_2UKeO2pW8Mpf18A Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:06:32.033827 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: b997560479c642e0ac43c0b08a1bf157 Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 1f648ad9-e7ac-40a4-9901-27d437beb2db could not be found. Sep 18 07:06:32.035035 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '1f648ad9-e7ac-40a4-9901-27d437beb2db'} Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1f648ad9-e7ac-40a4-9901-27d437beb2db could not be found. Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:06:32.035687 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1f648ad9-e7ac-40a4-9901-27d437beb2db could not be found. Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1f648ad9-e7ac-40a4-9901-27d437beb2db could not be found. Sep 18 07:06:32.036967 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:32.038081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Returning 404 to user: Instance 1f648ad9-e7ac-40a4-9901-27d437beb2db could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:06:32.038081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-5c44756d-78f4-4543-930d-ff709640c95e tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db" status: 404 len: 111 microversion: 2.1 time: 0.030287 Sep 18 07:06:32.038081 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1760/2940] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:32 2022] GET /compute/v2.1/servers/1f648ad9-e7ac-40a4-9901-27d437beb2db => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:06:32.151876 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:32.152520 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:32.158934 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:32.159415 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:32.159852 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:32.160530 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:32.161013 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:32.161432 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:32.291349 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:32.299902 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-60155857-24fe-4e12-8c2b-b01b7599e2ea tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1313 microversion: 2.1 time: 0.152329 Sep 18 07:06:32.300686 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1181/2941] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:32 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1313 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:32.989423 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-746f08dd-edcf-476a-97fa-1a22e90bf72f req-d6206a7a-39b8-4dfa-935e-eb4efdff39e3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "623c2940-94eb-4476-81e7-6e0a20b6414b", "name": "network-vif-plugged", "status": "completed", "tag": "80aebc1c-91c6-4394-ab2d-9b8046186ebc"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:32.999956 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-746f08dd-edcf-476a-97fa-1a22e90bf72f req-d6206a7a-39b8-4dfa-935e-eb4efdff39e3 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:33.000476 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-746f08dd-edcf-476a-97fa-1a22e90bf72f req-d6206a7a-39b8-4dfa-935e-eb4efdff39e3 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:33.001058 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-746f08dd-edcf-476a-97fa-1a22e90bf72f req-d6206a7a-39b8-4dfa-935e-eb4efdff39e3 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:33.012255 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-746f08dd-edcf-476a-97fa-1a22e90bf72f req-d6206a7a-39b8-4dfa-935e-eb4efdff39e3 service nova] Creating event network-vif-plugged:80aebc1c-91c6-4394-ab2d-9b8046186ebc for instance 623c2940-94eb-4476-81e7-6e0a20b6414b on hv-cinder-85535 Sep 18 07:06:33.017145 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-746f08dd-edcf-476a-97fa-1a22e90bf72f req-d6206a7a-39b8-4dfa-935e-eb4efdff39e3 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031509 Sep 18 07:06:33.017911 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1761/2942] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:06:33.323030 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:33.323644 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:33.329884 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:33.330364 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:33.330833 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:33.331512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:33.331959 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:33.332382 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:33.467651 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:33.475277 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e4d5f3b2-f6b2-4be6-a5a0-e7238dfebcef tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1313 microversion: 2.1 time: 0.156017 Sep 18 07:06:33.475835 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1182/2943] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:33 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:34.499455 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:34.500152 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:34.506886 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:34.507372 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:34.507810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:34.508536 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:34.508991 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:34.509418 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:34.697677 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:34.705640 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-d0232e02-2256-4113-9e53-ed6fefaaf07f tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1496 microversion: 2.1 time: 0.210370 Sep 18 07:06:34.706211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1762/2944] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:34 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1496 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:35.151917 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:35.152451 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:35.154790 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=2 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:35.155446 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell0:SELECT=6 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:35.161484 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.162057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.162568 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.163430 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.163938 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.164435 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.191629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Going to try to terminate instance {{(pid=120380) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 07:06:35.208800 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:35.209493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:35.217504 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell0:SELECT=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:35.220084 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.220738 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.221303 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.222160 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.222803 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.223250 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.227464 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG dbcounter [-] [120380] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:35.252608 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Going to try to terminate instance {{(pid=120381) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 07:06:35.284527 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:35.292699 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-d79e54c3-6a11-4c86-839d-c36f30e23033 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "DELETE /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 204 len: 0 microversion: 2.1 time: 0.143609 Sep 18 07:06:35.296983 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1183/2945] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 07:06:35 2022] DELETE /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 07:06:35.303804 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-ff3d59f7-0120-4319-90c5-554f804cc50b req-c6eeeabc-5dce-4767-b723-3c67658f0e38 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4726cd7a-c664-4498-a510-2fffeb964af3", "tag": "3d68cbf6-c73d-4226-848b-9c00e71a04b4"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:35.313254 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ff3d59f7-0120-4319-90c5-554f804cc50b req-c6eeeabc-5dce-4767-b723-3c67658f0e38 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.313725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ff3d59f7-0120-4319-90c5-554f804cc50b req-c6eeeabc-5dce-4767-b723-3c67658f0e38 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.314150 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-ff3d59f7-0120-4319-90c5-554f804cc50b req-c6eeeabc-5dce-4767-b723-3c67658f0e38 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.326339 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-ff3d59f7-0120-4319-90c5-554f804cc50b req-c6eeeabc-5dce-4767-b723-3c67658f0e38 service nova] Creating event network-changed:3d68cbf6-c73d-4226-848b-9c00e71a04b4 for instance 4726cd7a-c664-4498-a510-2fffeb964af3 on hv-cinder-85535 Sep 18 07:06:35.331467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-ff3d59f7-0120-4319-90c5-554f804cc50b req-c6eeeabc-5dce-4767-b723-3c67658f0e38 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032989 Sep 18 07:06:35.332228 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1184/2946] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:06:35.337820 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:35.338521 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:35.344940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.345428 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.345856 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.346546 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.347021 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.347440 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.356105 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-20e387cf-9549-4015-8c20-b5d05ebea4f0 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "DELETE /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 204 len: 0 microversion: 2.1 time: 0.150948 Sep 18 07:06:35.360113 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1763/2947] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 07:06:35 2022] DELETE /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 18 07:06:35.370907 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:35.371643 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:35.379211 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.379717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.380314 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.381038 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:35.381502 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:35.381965 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:35.501424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:35.509967 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-8ec560e4-f773-4a8f-b4bb-42cd521f4d69 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1502 microversion: 2.1 time: 0.176694 Sep 18 07:06:35.510792 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1185/2948] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:35 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1502 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:35.538002 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:35.546108 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-c0dc0094-ecda-44f9-9cdb-35a333139cf9 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1578 microversion: 2.1 time: 0.184305 Sep 18 07:06:35.546757 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1764/2949] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:35 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1578 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:36.532939 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:36.533553 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:36.540576 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:36.541052 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:36.541483 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:36.542160 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:36.542634 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:36.543056 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:36.569278 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:36.569998 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:36.577030 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:36.577523 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:36.577962 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:36.578678 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:36.579132 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:36.579556 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:36.676875 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:36.685258 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-05b69fa0-cb44-4581-a920-1d2f92261640 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1502 microversion: 2.1 time: 0.156207 Sep 18 07:06:36.685904 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1186/2950] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:36 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1502 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:36.712062 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:36.719858 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-97b6b712-c3b4-4535-9957-bde01b10f535 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1578 microversion: 2.1 time: 0.154390 Sep 18 07:06:36.720361 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1765/2951] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:36 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1578 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:06:37.708512 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:37.709122 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:37.716016 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:37.716584 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:37.717019 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:37.717714 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:37.718165 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:37.718598 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:37.745555 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:37.748498 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:37.759997 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:37.760496 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:37.760935 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:37.761642 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:37.762094 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:37.762519 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:37.972407 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.policy [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9842848655fe479bbea42fd0f2ca4ffa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '523fb59392af43bcb2be01e17406cef7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120380) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:37.981523 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-c6b48231-85c0-439c-9f58-98d8e4decb8a tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 200 len: 1502 microversion: 2.1 time: 0.277022 Sep 18 07:06:37.983044 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1187/2952] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:37 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 1502 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:38.057128 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:38.065787 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-81f47c6c-063c-4951-95df-883b1703d3d6 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1578 microversion: 2.1 time: 0.326087 Sep 18 07:06:38.074717 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1766/2953] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:37 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1578 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 07:06:38.317680 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [req-97aee214-c398-4db6-b116-305abac42617 req-450cb2fc-1a9b-41ec-9517-a9a1a79de8c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4726cd7a-c664-4498-a510-2fffeb964af3", "name": "network-vif-deleted", "tag": "3d68cbf6-c73d-4226-848b-9c00e71a04b4"}]} {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:38.328565 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-97aee214-c398-4db6-b116-305abac42617 req-450cb2fc-1a9b-41ec-9517-a9a1a79de8c1 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:38.329194 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-97aee214-c398-4db6-b116-305abac42617 req-450cb2fc-1a9b-41ec-9517-a9a1a79de8c1 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:38.329471 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [req-97aee214-c398-4db6-b116-305abac42617 req-450cb2fc-1a9b-41ec-9517-a9a1a79de8c1 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:38.342127 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.compute.server_external_events [req-97aee214-c398-4db6-b116-305abac42617 req-450cb2fc-1a9b-41ec-9517-a9a1a79de8c1 service nova] Creating event network-vif-deleted:3d68cbf6-c73d-4226-848b-9c00e71a04b4 for instance 4726cd7a-c664-4498-a510-2fffeb964af3 on hv-cinder-85535 Sep 18 07:06:38.347525 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [req-97aee214-c398-4db6-b116-305abac42617 req-450cb2fc-1a9b-41ec-9517-a9a1a79de8c1 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033496 Sep 18 07:06:38.348695 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1188/2954] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 07:06:38.393335 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [req-4bf78204-ed1c-4c24-8545-b7fbb508de8c req-f83aa9e1-be7f-449f-8bb1-4cb2b5c7e7e9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "623c2940-94eb-4476-81e7-6e0a20b6414b", "name": "network-vif-deleted", "tag": "80aebc1c-91c6-4394-ab2d-9b8046186ebc"}]} {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 07:06:38.404740 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4bf78204-ed1c-4c24-8545-b7fbb508de8c req-f83aa9e1-be7f-449f-8bb1-4cb2b5c7e7e9 service nova] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:38.405445 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4bf78204-ed1c-4c24-8545-b7fbb508de8c req-f83aa9e1-be7f-449f-8bb1-4cb2b5c7e7e9 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:38.405863 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [req-4bf78204-ed1c-4c24-8545-b7fbb508de8c req-f83aa9e1-be7f-449f-8bb1-4cb2b5c7e7e9 service nova] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:38.418877 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.compute.server_external_events [req-4bf78204-ed1c-4c24-8545-b7fbb508de8c req-f83aa9e1-be7f-449f-8bb1-4cb2b5c7e7e9 service nova] Creating event network-vif-deleted:80aebc1c-91c6-4394-ab2d-9b8046186ebc for instance 623c2940-94eb-4476-81e7-6e0a20b6414b on hv-cinder-85535 Sep 18 07:06:38.425550 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [req-4bf78204-ed1c-4c24-8545-b7fbb508de8c req-f83aa9e1-be7f-449f-8bb1-4cb2b5c7e7e9 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035898 Sep 18 07:06:38.426252 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1767/2955] 10.222.0.20 () {64 vars in 1317 bytes} [Sun Sep 18 07:06:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 07:06:39.006441 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:39.007109 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] [instance: 623c2940-94eb-4476-81e7-6e0a20b6414b] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:39.013163 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:39.013647 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:39.014071 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:39.014786 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:39.015234 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:39.015650 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:39.028469 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] HTTP exception thrown: Instance 623c2940-94eb-4476-81e7-6e0a20b6414b could not be found. Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Request method failure captured: Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b HTTP/1.1 Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJsNhnklrO1ENM6wCDtESPdZQ4Ch9WcJ9N318z0zTaT-rxM4HbMt6UrkrFU_gnrfqBqSRud0WYeacdNSJPSHNwt-lQM1kF8nKHjH0thKMhHAAekWqhy5QPlnD62DySRxYS4vFMPYip0thECHFE-aH25sjdgcmELnoWKgtMtriYbFmfJ0 Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: 523fb59392af43bcb2be01e17406cef7 Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-VolumesActionsTest-759759597 Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 07:06:39.030068 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/523fb59392af43bcb2be01e17406cef7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_523fb59392af43bcb2be01e17406cef7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/523fb59392af43bcb2be01e17406cef7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/523fb59392af43bcb2be01e17406cef7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-VolumesActionsTest-759759597 Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: 523fb59392af43bcb2be01e17406cef7 Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-VolumesActionsTest-759759597 Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-VolumesActionsTest-759759597-project Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: 9842848655fe479bbea42fd0f2ca4ffa Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-VolumesActionsTest-759759597-project Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 623c2940-94eb-4476-81e7-6e0a20b6414b could not be found. Sep 18 07:06:39.031487 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '623c2940-94eb-4476-81e7-6e0a20b6414b'} Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 623c2940-94eb-4476-81e7-6e0a20b6414b could not be found. Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:06:39.032337 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 623c2940-94eb-4476-81e7-6e0a20b6414b could not be found. Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 623c2940-94eb-4476-81e7-6e0a20b6414b could not be found. Sep 18 07:06:39.033719 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:39.034940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] Returning 404 to user: Instance 623c2940-94eb-4476-81e7-6e0a20b6414b could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:06:39.034940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-e5737ea5-0577-41a6-b3d9-50d8c5070dc1 tempest-VolumesActionsTest-759759597 tempest-VolumesActionsTest-759759597-project] 10.222.0.20 "GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b" status: 404 len: 111 microversion: 2.1 time: 0.029941 Sep 18 07:06:39.034940 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1189/2956] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:39 2022] GET /compute/v2.1/servers/623c2940-94eb-4476-81e7-6e0a20b6414b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:06:39.091118 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:39.091822 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:39.098656 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:39.098976 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:39.099427 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:39.100125 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:39.100582 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:39.101006 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:39.145273 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:39.145966 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:39.146406 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:39.146879 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.objects.instance [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lazy-loading 'fault' on Instance uuid 4726cd7a-c664-4498-a510-2fffeb964af3 {{(pid=120381) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 07:06:39.190195 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.policy [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b997560479c642e0ac43c0b08a1bf157', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f707372e1ec948dcb7e89b9a046cccfc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120381) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 07:06:39.198021 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-6c76b7e5-66a1-4b4c-a787-96d77c8ed14c tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 200 len: 1229 microversion: 2.1 time: 0.111063 Sep 18 07:06:39.199575 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1768/2957] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:39 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 1229 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 07:06:40.221419 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:40.222039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 4726cd7a-c664-4498-a510-2fffeb964af3] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:40.228543 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:40.229031 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:40.229467 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:40.230153 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:40.230629 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:40.231057 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:40.244764 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] HTTP exception thrown: Instance 4726cd7a-c664-4498-a510-2fffeb964af3 could not be found. Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Request method failure captured: Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 HTTP/1.1 Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJsHwP6HAV62fkLOiQ3vPuoYHfp1Z9xkdqSmTuwfMY_LHM9wYfsLVFiPbBq7p9R7RfpiZ5fmqUkmEHIg40p79sRdnZjKKH9eyr6xj9C_mgNAVmfdnPagvZ8Rcy4eTVfBiMeAcxbzHLVFgirfm03gJXVA-RewKu78_2UKeO2pW8Mpf18A Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 07:06:40.246263 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: b997560479c642e0ac43c0b08a1bf157 Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 4726cd7a-c664-4498-a510-2fffeb964af3 could not be found. Sep 18 07:06:40.247666 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '4726cd7a-c664-4498-a510-2fffeb964af3'} Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4726cd7a-c664-4498-a510-2fffeb964af3 could not be found. Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:06:40.248463 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4726cd7a-c664-4498-a510-2fffeb964af3 could not be found. Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4726cd7a-c664-4498-a510-2fffeb964af3 could not be found. Sep 18 07:06:40.249725 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.250903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Returning 404 to user: Instance 4726cd7a-c664-4498-a510-2fffeb964af3 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:06:40.250903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-a26b46db-80c8-47fd-b1e2-4b08d81c0523 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3" status: 404 len: 111 microversion: 2.1 time: 0.030177 Sep 18 07:06:40.250903 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1190/2958] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:40 2022] GET /compute/v2.1/servers/4726cd7a-c664-4498-a510-2fffeb964af3 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:06:40.683493 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:40.684000 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.compute.api [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120381) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:40.690369 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:40.690900 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:40.691342 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:40.692028 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:40.692489 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:40.692915 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG oslo_concurrency.lockutils [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120381) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:40.706505 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.wsgi [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] HTTP exception thrown: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Request method failure captured: Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: request: DELETE /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 HTTP/1.1 Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept: application/json Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Accept-Encoding: identity Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Connection: close Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Length: 0 Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Content-Type: application/json Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Host: 10.222.0.20 Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: Openstack-System-Scope: None Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: User-Agent: python-urllib3/1.26.12 Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Auth-Token: gAAAAABjJsHwP6HAV62fkLOiQ3vPuoYHfp1Z9xkdqSmTuwfMY_LHM9wYfsLVFiPbBq7p9R7RfpiZ5fmqUkmEHIg40p79sRdnZjKKH9eyr6xj9C_mgNAVmfdnPagvZ8Rcy4eTVfBiMeAcxbzHLVFgirfm03gJXVA-RewKu78_2UKeO2pW8Mpf18A Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Id: None Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Domain-Name: None Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Identity-Status: Confirmed Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Is-Admin-Project: True Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Id: default Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Domain-Name: Default Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Project-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Role: reader,member Sep 18 07:06:40.708163 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Roles: reader,member Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-Tenant-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Id: default Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Domain-Name: Default Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Id: b997560479c642e0ac43c0b08a1bf157 Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: X-User-Name: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: method: > Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: exception: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.709511 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: action_args: {'id': '5cdcc319-ec7c-4d46-a87f-996f83c03821'} Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:06:40.710301 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.711415 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.712469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Returning 404 to user: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. {{(pid=120381) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:06:40.712469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-9e35ccdb-244b-43f6-a320-aa9393762b63 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "DELETE /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 404 len: 111 microversion: 2.1 time: 0.030437 Sep 18 07:06:40.712469 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1769/2959] 10.222.0.20 () {58 vars in 1239 bytes} [Sun Sep 18 07:06:40 2022] DELETE /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:06:40.718039 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '>' {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:40.718679 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.compute.api [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] [instance: 5cdcc319-ec7c-4d46-a87f-996f83c03821] Fetching instance by UUID {{(pid=120380) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 07:06:40.724517 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:40.724993 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:40.725424 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:40.726099 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Acquiring lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 07:06:40.726544 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 07:06:40.726991 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG oslo_concurrency.lockutils [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Lock "63c4bc6f-f107-41b9-b2c1-7478fed24e9a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120380) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 07:06:40.739819 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.wsgi [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] HTTP exception thrown: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Request method failure captured: Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: request: GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 HTTP/1.1 Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept: application/json Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Accept-Encoding: identity Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Connection: close Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Length: 0 Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Content-Type: application/json Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Host: 10.222.0.20 Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: Openstack-System-Scope: None Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: User-Agent: python-urllib3/1.26.12 Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Auth-Token: gAAAAABjJsHwP6HAV62fkLOiQ3vPuoYHfp1Z9xkdqSmTuwfMY_LHM9wYfsLVFiPbBq7p9R7RfpiZ5fmqUkmEHIg40p79sRdnZjKKH9eyr6xj9C_mgNAVmfdnPagvZ8Rcy4eTVfBiMeAcxbzHLVFgirfm03gJXVA-RewKu78_2UKeO2pW8Mpf18A Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Id: None Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Domain-Name: None Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Identity-Status: Confirmed Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Is-Admin-Project: True Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Id: default Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Domain-Name: Default Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Project-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Role: reader,member Sep 18 07:06:40.741305 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Roles: reader,member Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f707372e1ec948dcb7e89b9a046cccfc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}] Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Id: f707372e1ec948dcb7e89b9a046cccfc Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-Tenant-Name: tempest-TestVolumeBootPattern-548700184 Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Id: default Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Domain-Name: Default Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Id: b997560479c642e0ac43c0b08a1bf157 Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: X-User-Name: tempest-TestVolumeBootPattern-548700184-project Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: method: > Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: exception: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.743462 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: action_args: {'id': '5cdcc319-ec7c-4d46-a87f-996f83c03821'} Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: {{(pid=120380) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 07:06:40.744315 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. Sep 18 07:06:40.745754 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: ERROR nova.api.openstack.wsgi Sep 18 07:06:40.747283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: DEBUG nova.api.openstack.wsgi [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Returning 404 to user: Instance 5cdcc319-ec7c-4d46-a87f-996f83c03821 could not be found. {{(pid=120380) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 07:06:40.747283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: INFO nova.api.openstack.requestlog [None req-7019bcd2-52b7-4fe7-aa4e-bd3f921c3477 tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821" status: 404 len: 111 microversion: 2.1 time: 0.028421 Sep 18 07:06:40.747283 dv-cinder-855359-3-iscsi devstack@n-api.service[120380]: [pid: 120380|app: 0|req: 1191/2960] 10.222.0.20 () {58 vars in 1236 bytes} [Sun Sep 18 07:06:40 2022] GET /compute/v2.1/servers/5cdcc319-ec7c-4d46-a87f-996f83c03821 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 07:06:44.181449 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG nova.api.openstack.wsgi [None req-842443f3-68c7-4f96-9787-86b26879509b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] Calling method '.version_select at 0x7fe6701db700>' {{(pid=120381) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 07:06:44.190810 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: DEBUG dbcounter [-] [120381] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=2 {{(pid=120381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:44.211676 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: INFO nova.api.openstack.requestlog [None req-842443f3-68c7-4f96-9787-86b26879509b tempest-TestVolumeBootPattern-548700184 tempest-TestVolumeBootPattern-548700184-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-671757144" status: 202 len: 0 microversion: 2.1 time: 0.034056 Sep 18 07:06:44.215487 dv-cinder-855359-3-iscsi devstack@n-api.service[120381]: [pid: 120381|app: 0|req: 1770/2961] 10.222.0.20 () {58 vars in 1260 bytes} [Sun Sep 18 07:06:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-671757144 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)